Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th June 2022, 14:31   #19521  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 220
Quote:
Originally Posted by Ryushin View Post
This is why you have to tune the CPU threads for each of your encoding Servers (per computer). So run a 1 minute chunk and time each one just using MDegrain2. Adjust the threads until you find the sweet spot for the encoding server. I would still use the Limit to the following filters option enabled though.

I think if you spend a couple of hours or so tuning your threads for each of your Servers it will pay big dividends in encoding time vs watts used.

Atak: You might want to create a tool that uses MDegrain2/SMDegrain that will find the sweet spot for thread use per Computer. As CPU core count seems to be increasing more and more people are going to hit this issue.
I will try this other procedure when I get a chance, but this "problem" only seems to be A "problem" when using very heavy filters, other wise, it's working great, and CPU utilization is basically 100%.

Even when it's encoding very slowly, it's still pushing the CPU's to pretty much 100%, can't ask anymore than that.

Correct me if I'm wrong, BUT all the encoding servers use the RipBot settings of the client machine, so that individual server settings "do not apply" !!!

As Atak mentioned once, in reality the server PC's ONLY need the Encoding Server, not the whole "app", you'd only need to manually start the Encoding Server, and DE will do the rest, so there's no settings required.

But it IS strange that in some situations, a 12 core system, does a faster job than a 16 core system....
__________________
We all need to get a life, before it's over.

Long term RipBot264 user.

Ryzen 5950X, 5900X & 3950X.
Dual Xeon E5-2697v2.
2 x Dell Poweredge R720's (dual E5-2680v2)
and many others

Last edited by TDS; 12th June 2022 at 14:40.
TDS is offline   Reply With Quote
Old 12th June 2022, 15:27   #19522  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,658
You can manually define number of prefetch threads for each server like this
Code:
EncodingServer.exe /avisynth-prefetch-threads 8
By the default prefetch threads = number of physical cores on current PC

Last edited by Atak_Snajpera; 12th June 2022 at 15:30.
Atak_Snajpera is offline   Reply With Quote
Old 13th June 2022, 02:00   #19523  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 220
Quote:
Originally Posted by Atak_Snajpera View Post
You can manually define number of prefetch threads for each server like this
Code:
EncodingServer.exe /avisynth-prefetch-threads 8
By the default prefetch threads = number of physical cores on current PC
Thanks for chiming in, Atak...

You know, that option has been there all along, staring us in the face, but not really knowing what it does.

So, under "Settings", "Distributed Encoding" tab, click on the little "Spanner", then "Command line help".....

EDIT:-

I just tried this on my 3950X 16 Core (which is currently running as an encoding server), and adding that command, only using 24 instead of 8, hasn't changed anything, that I can detect.

In the Encoding Server "window", it still shows :- x265 [info]: Thread pool created using 32 threads

So again, ALL the server machines take their "instructions" and "settings" from the main Client PC, regardless how the server PC is setup !


EDIT #2:-

OK, changed the command line on the Client PC, and even tho it states that Avisynth is only using 24 threads, the x265 info remains @ what ever the CPU provides.

The ONLY way it seems, to change the amount of threads that x265 uses is in the command line for x265, by adding --pools [integer]
__________________
We all need to get a life, before it's over.

Long term RipBot264 user.

Ryzen 5950X, 5900X & 3950X.
Dual Xeon E5-2697v2.
2 x Dell Poweredge R720's (dual E5-2680v2)
and many others

Last edited by TDS; 13th June 2022 at 03:05.
TDS is offline   Reply With Quote
Old 13th June 2022, 12:20   #19524  |  Link
Ryushin
Registered User
 
Join Date: Mar 2011
Posts: 349
Quote:
Originally Posted by TDS View Post
Even when it's encoding very slowly, it's still pushing the CPU's to pretty much 100%, can't ask anymore than that.
But it IS strange that in some situations, a 12 core system, does a faster job than a 16 core system....
Yea, I saw a 100% CPU as well when I had unlimited threads. A bottleneck is still taking place. By optimizing the threads, you are removing the bottleneck that was causing slower processing.
Ryushin is offline   Reply With Quote
Old 13th June 2022, 16:43   #19525  |  Link
Ryushin
Registered User
 
Join Date: Mar 2011
Posts: 349
RipBot 2.0

Hi Atak,

If your schedule allows, it would be nice to get a RipBot 2.0. You might want to change the name to just RipBot or maybe something like RipBotEncoder.

SMDegrain has been a game changer in quality, though not without its steeper learning curve. I have transitioned over to SMDegrain exclusively though using Pauly Dunnes Tools. Though I create my own scripts to use the CPU only instead of the GPU. SMDegrain is powerful and flexible enough to be the primary degrainer for RB. The options for Blur plus Contrasharp on very grainy sources produces output that defies what I would think would be possible.

Dolby Vision is also easier to deal with and is scriptable: https://forum.doom9.org/showthread.php?t=183479 There are some options that have to be passed if cropping is going to be used in the final output.

It would be nice to see a UI overhaul of the AVI Synth settings. If the window could be made quite a bit larger to see the 2.40:1 bottom borders for 4K sources like you can with HD sources. The larger window will also give you more space for the options. For the SMDegrain settings. A pull down menu with a few presets for most people, something like this:
Light: SMDegrain(video,tr=2,thSAD=200,thSADC=100,refinemotion=true)
Medium: SMDegrain(video,tr=4,thSAD=400,thSADC=200,refinemotion=true)
Heavy: SMDegrain(video,tr=8,thSAD=800,thSADC=400,refinemotion=true)
Then add a Blur+Sharp radio button next to SMDegrain filter that adds the following depending how heavy they wanted it:
Light: contrasharp=true,prefilter=0
Medium: contrasharp=true,prefilter=1
Heavy: contrasharp=true,prefilter=2
Then having an Advanced option on the bottom of the pull down that opens up another new window with more switches and options. Such as having the tr, thSAD (I think thSADC should always be half of thSAD), refinemotion, contrasharp, prefilter, str (for darker sources), blksize, etc. A little "?" mark next to each option would be nice with a mouse hover pop up that provides details on that option.

Changes for the Distributed Encoding model would be good too. The Primary server would dole out work to the clients. The RipBot install on the client would have something like a "DE Client" startup option and then RB will just start up minimized to the system tray with no Server processes started. The Primary RB server would then communicate with the RB Client to start up how many server processes along with the options for those server processes. The DE window can also have a per Client Thread Calibration button that you can click that will open up a window with two grainy 1 minutes sources, one for HD and one for 4K that the user can choose select one they want, then pop up a warning that this could take a long time, then the Primary server will send the Job to the RB Client, start up a single Server Process along with the number of threads to test for SMDegrain Heavy or MDegrain2-400, over and over until the optimum number of threads is found. Though some intelligence can be added, like start with half threads, then the lower quarter threads, upper quarter of threads, then fine tune it between the two quarters that had the best speed.

Atak, I know I've asked this before, but it would be nice to get RB up on Git along with the source and a issue tracker.
Ryushin is offline   Reply With Quote
Old 14th June 2022, 04:06   #19526  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 220
Quote:
Originally Posted by Ryushin View Post
Yea, I saw a 100% CPU as well when I had unlimited threads. A bottleneck is still taking place. By optimizing the threads, you are removing the bottleneck that was causing slower processing.
Hi, are you reading everything I've said ??

It's ONLY happening with the VERY very strong filter I am using, on ALL the other filters, they are all pretty much equal encoding speed.

So there must be something going on when you bump up the "tr", "thSAD" & "thSADC" levels/numbers.

Up to "tr=8" is fine...

But one thing I need to repeat, is, ALL the server PC's, regardless of any tweaks & tuning, take ALL their settings & instructions from the client PC.

So if you went to all the trouble of tuning every server, when running them as a client, is all over ridden, when they are used as a server, because the servers ONLY require the Encoding Server to do the DE process.

Now I'm pretty sure that running RipBot on a VM, wouldn't change the fundamentals, and I don't use VM's for ANYTHING !!

But there's obviously a significant difference between your hardware & setup, to what I have.

I have tried everything that has been suggested, and have not seen any significant change, so I think we need to put this topic to rest, it's just going around in circles.
__________________
We all need to get a life, before it's over.

Long term RipBot264 user.

Ryzen 5950X, 5900X & 3950X.
Dual Xeon E5-2697v2.
2 x Dell Poweredge R720's (dual E5-2680v2)
and many others
TDS is offline   Reply With Quote
Old 14th June 2022, 04:48   #19527  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 643
Quote:
Originally Posted by Ryushin View Post

SMDegrain has been a game changer in quality, though not without its steeper learning curve. I have transitioned over to SMDegrain exclusively though using Pauly Dunnes Tools. Though I create my own scripts to use the CPU only instead of the GPU. SMDegrain is powerful and flexible enough to be the primary degrainer for RB. The options for Blur plus Contrasharp on very grainy sources produces output that defies what I would think would be possible.
WOW, I'm blown away with this post !!!

I knew when I started to implement SMDegrain into RipBot, that is was superior, and that was before I realised that there were options to use pre-filters as well.

It wasn't until I discovered Dogway's Doom9 page, that I realised all this.

https://forum.doom9.org/showthread.php?t=182881

I certainly can't take any credit for what SMDegrain can do !!!

But I have to say that there are a lot more pre-filters, than just 0,1 & 2, sure there are GPU based pre-filters that other users could benefit from.

There also many other filters available in my builds, too, not just focusing on SMDegrain.

And finally, since I first post my RipBot build(s), there has been 100's of downloads, the latest build posted only a week ago, has been downloaded over 30 times.

There has only been a few say thanks, but for Ryushin to say these words, is pretty amazing.
__________________
Not poorly done, just doin' it my way !!!
So much to do, and so little time :(
Pauly Dunne is offline   Reply With Quote
Old 20th June 2022, 04:26   #19528  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 643
Well, here we go again, another update...

Updated several of the SMDegrain based filters.

I have also added several Debanding Filters.

I have been doing some more tidying up of the Script & Custom folders.

https://www.mediafire.com/file/cml44...06-22).7z/file
__________________
Not poorly done, just doin' it my way !!!
So much to do, and so little time :(
Pauly Dunne is offline   Reply With Quote
Old 22nd June 2022, 17:13   #19529  |  Link
reallywonder
Registered User
 
Join Date: Jun 2022
Posts: 2
Hi, first-time poster here. Been using this since about 2009 or so, love it.

I'm wondering if there's an option in the .ini for disabling the balloon notifications when encoding is done? I've searched the thread and didn't see anyone ask this before.

I auto-hide taskbar at top of screen, and the balloon makes the taskbar appear and is annoying, and sometimes, even after clearing the balloon, the taskbar "sticks" and doesn't go away even though it should (this is a MS problem).

I didn't see anything in the UI settings, and the .ini only really has stuff in there if something has been changed from default.
reallywonder is offline   Reply With Quote
Old 22nd June 2022, 18:37   #19530  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,658
Quote:
Originally Posted by reallywonder View Post
Hi, first-time poster here. Been using this since about 2009 or so, love it.

I'm wondering if there's an option in the .ini for disabling the balloon notifications when encoding is done? I've searched the thread and didn't see anyone ask this before.

I auto-hide taskbar at top of screen, and the balloon makes the taskbar appear and is annoying, and sometimes, even after clearing the balloon, the taskbar "sticks" and doesn't go away even though it should (this is a MS problem).

I didn't see anything in the UI settings, and the .ini only really has stuff in there if something has been changed from default.
I will add that option in next version
Atak_Snajpera is offline   Reply With Quote
Old 23rd June 2022, 01:42   #19531  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 220
Quote:
Originally Posted by reallywonder View Post
Hi, first-time poster here. Been using this since about 2009 or so, love it.

I'm wondering if there's an option in the .ini for disabling the balloon notifications when encoding is done? I've searched the thread and didn't see anyone ask this before.

I auto-hide taskbar at top of screen, and the balloon makes the taskbar appear and is annoying, and sometimes, even after clearing the balloon, the taskbar "sticks" and doesn't go away even though it should (this is a MS problem).

I didn't see anything in the UI settings, and the .ini only really has stuff in there if something has been changed from default.
Welcome to the Forum...

You know, I have been using RipBot a lot longer than that, and I cannot relate to this "popup" problem.

What version of Windows are you using ??
__________________
We all need to get a life, before it's over.

Long term RipBot264 user.

Ryzen 5950X, 5900X & 3950X.
Dual Xeon E5-2697v2.
2 x Dell Poweredge R720's (dual E5-2680v2)
and many others
TDS is offline   Reply With Quote
Old 23rd June 2022, 01:44   #19532  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 220
Quote:
Originally Posted by Atak_Snajpera View Post
I will add that option in next version
Wow, "next version", I'm REALLY looking forward to this, I hope it's soon.
__________________
We all need to get a life, before it's over.

Long term RipBot264 user.

Ryzen 5950X, 5900X & 3950X.
Dual Xeon E5-2697v2.
2 x Dell Poweredge R720's (dual E5-2680v2)
and many others
TDS is offline   Reply With Quote
Old 23rd June 2022, 05:35   #19533  |  Link
reallywonder
Registered User
 
Join Date: Jun 2022
Posts: 2
Quote:
Originally Posted by TDS View Post
Welcome to the Forum...

You know, I have been using RipBot a lot longer than that, and I cannot relate to this "popup" problem.

What version of Windows are you using ??
still using 7, lol. not moving up to 10/11.
reallywonder is offline   Reply With Quote
Old 23rd June 2022, 06:11   #19534  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 220
Quote:
Originally Posted by reallywonder View Post
still using 7, lol. not moving up to 10/11.
OK, well that explains why I don't see it, I haven't used W7 for a L-O-N-G time.

RB works well with W10, but there a couple of small issues when using W11, but it all still works, in my experience, anyway.

I'm using W11 Workstation Pro.

Hopefully the "next version" will address a few things..
__________________
We all need to get a life, before it's over.

Long term RipBot264 user.

Ryzen 5950X, 5900X & 3950X.
Dual Xeon E5-2697v2.
2 x Dell Poweredge R720's (dual E5-2680v2)
and many others
TDS is offline   Reply With Quote
Old 26th June 2022, 02:17   #19535  |  Link
NiGHTsC
Registered User
 
Join Date: May 2007
Posts: 85
One quick and new AVS user
How do you rotate a video now?
I tried Turn180() in AviSynth but it gives error...
NiGHTsC is offline   Reply With Quote
Old 26th June 2022, 04:48   #19536  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 220
Quote:
Originally Posted by NiGHTsC View Post
One quick and new AVS user
How do you rotate a video now?
I tried Turn180() in AviSynth but it gives error...
I actually tried this last week, what you need to do is:-

Code:
video=Turn180(video)
__________________
We all need to get a life, before it's over.

Long term RipBot264 user.

Ryzen 5950X, 5900X & 3950X.
Dual Xeon E5-2697v2.
2 x Dell Poweredge R720's (dual E5-2680v2)
and many others
TDS is offline   Reply With Quote
Old 26th June 2022, 06:20   #19537  |  Link
NiGHTsC
Registered User
 
Join Date: May 2007
Posts: 85
Quote:
Originally Posted by TDS View Post
I actually tried this last week, what you need to do is:-

Code:
video=Turn180(video)
Thank you Thank you Thank you!!!
NiGHTsC is offline   Reply With Quote
Reply

Tags
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:49.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.