RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY [Archive] - Page 390

guest

4th May 2022, 14:55

Thank you. I haven't had any issues or need to post anything. So I can use this MKVToolNix to add files to my AVCHD once it's done in RipBot?

You certainly should be able to...the main movie file in an AVCHD if I recall is an .m2ts... that should load into MKVToolNix, you can then add / remove other audio & subtitle tracks, (simply drag n drop) and let it do it's thing.

You can use the version within RipBot264 Tools folder (but it's old), so I would suggest you go here :- https://www.fosshub.com/MKVToolNix.html & grab the "installer" MKVToolNix Windows installer (64-bit), and you're set.

edit:- I just realised that this will produce an .mkv file, if you want to maintain the .m2ts format for the AVCHD, you will need to use Tsmuxer to add the subtitle track(s).

Get the latest here :- https://www.videohelp.com/software/tsMuxeR

I have very rarely used RipBot to "do" subtitles !!!!

Atak_Snajpera

4th May 2022, 21:56

https://imgur.com/5gk0xg3

Tried adding this before but nothing was appearing.

The screenshot of the EncodingServer "Main" didn't save unfortunately.

But it was as if the encoding was done at the bottom would be for example 1466/1466 but still in "Encoding" mode and the TCP window be repeating 1462/1466.

I would like to see screenshot of process hacker showing what processes are still in memory when this issue appears.

alexflin

6th May 2022, 20:19

Hi, I need help, I'm using this program https://prnt.sc/kFn4wSDzIMNh to encode my videos getting UHD compatibility, can I add the same commands to ripbot264 to encode multiple videos in batch mode?

guest

7th May 2022, 10:50

Hi, I need help, I'm using this program https://prnt.sc/kFn4wSDzIMNh to encode my videos getting UHD compatibility, can I add the same commands to ripbot264 to encode multiple videos in batch mode?

Welcome,

So what app is that screen shot of ??

You might be able to create your own custom x265 "profile" within RipBot, and add some of those commands shown, but it might be a bit "hit and miss" until you get what you want.

For info on command options go here :-

https://x265.readthedocs.io/en/master/cli.html

You can access the profile folder within the main RipBot264 folder, then go to the hevc folder, and you will find the default profiles.

RipBot is pretty good at figuring out what the source files are, and also x265 would figure out it's own settings, depending on the content.

As for HDR, RipBot handles that pretty well, but again, depending on the input content, you can't make something HDR, if it's not there to start with (afaik).

A lot of ppl convert HDR to SDR, which RipBot can do too.

I would suggest just trying a few options, and see how it goes, but there is ALWAY's room for improvement.

Good luck.

alexflin

8th May 2022, 01:20

Welcome,

So what app is that screen shot of ??

You might be able to create your own custom x265 "profile" within RipBot, and add some of those commands shown, but it might be a bit "hit and miss" until you get what you want.

For info on command options go here :-

https://x265.readthedocs.io/en/master/cli.html

You can access the profile folder within the main RipBot264 folder, then go to the hevc folder, and you will find the default profiles.

RipBot is pretty good at figuring out what the source files are, and also x265 would figure out it's own settings, depending on the content.

As for HDR, RipBot handles that pretty well, but again, depending on the input content, you can't make something HDR, if it's not there to start with (afaik).

A lot of ppl convert HDR to SDR, which RipBot can do too.

I would suggest just trying a few options, and see how it goes, but there is ALWAY's room for improvement.

Good luck.

Thanks for the help, this app is integrated in the Blu-Disc Studio HD, they recomend use this app to make compatible the video files with the program. I'm using the 30 days trial to create all the UDH blurays with menu of my favourite tv shows

guest

8th May 2022, 06:11

Hey,

Have been busy again updating most of the SMDegrain filter's, re-instated a few "older" filters.

Updates for a few "key" item's. etc, (changelog included)

Please note, this is a Tools folder replacement, so rename your existing Tools folder (just in case), and if you've modified any scripts.

ALL previous update links will be removed, as they are all in this build.

So go get it :-

The link that was here, is now obsolete, go here for the latest build..

https://forum.doom9.org/showpost.php?p=1969982&postcount=19510

And I would like to thank the 161 ppl that downloaded the previous Tools folder download back in February, and the many that also downloaded the 3 other update files.

And the few that provided very positive feedback :)

VirulentPip

10th May 2022, 13:27

Hey,

Have been busy again updating most of the SMDegrain filter's, re-instated a few "older" filters.

Updates for a few "key" item's. etc, (changelog included)

Please note, this is a Tools folder replacement, so rename your existing Tools folder (just in case), and if you've modified any scripts.

ALL previous update links will be removed, as they are all in this build.

So go get it :-

https://www.mediafire.com/file/1363bejqfy3gtsj/Tools+(08-05-22).7z/file

And I would like to thank the 161 ppl that downloaded the previous Tools folder download back in February, and the many that also downloaded the 3 other update files.

And the few that provided very positive feedback :)

Haven't updated with this latest one yet.. But since using your version, I've had 1 "incomplete" chunk in now well over 60 encodes maybe even more. Has made life easier knowing can just queue up a bunch over night knowing that the chances of them being completed by morning are now very very high. Thanks again.

Atak_Snajpera

10th May 2022, 15:14

Haven't updated with this latest one yet.. But since using your version, I've had 1 "incomplete" chunk in now well over 60 encodes maybe even more. Has made life easier knowing can just queue up a bunch over night knowing that the chances of them being completed by morning are now very very high. Thanks again.

You may try this workaround
EncodingServer.exe /restart-if-no-progress

Encoding will be restarted if there is no progress within 1 minute.

guest

11th May 2022, 00:47

Haven't updated with this latest one yet.. But since using your version, I've had 1 "incomplete" chunk in now well over 60 encodes maybe even more. Has made life easier knowing can just queue up a bunch over night knowing that the chances of them being completed by morning are now very very high. Thanks again.

Hi, that IS a very high success rate, I would be extremely happy with that :)

I use to encode overnight, back in the day, but nearly every time there would be a failure, while I was asleep, so I didn't bother doing that anymore, just wasn't worth it...

So thanks for the great feedback..

Cheers

guest

11th May 2022, 00:51

You may try this workaround
EncodingServer.exe /restart-if-no-progress

Encoding will be restarted if there is no progress within 1 minute.

Yeah, I'm afraid to say that this doesn't always work...I have been using this command for a long time, and if a chunk doesn't want to co-operate, it won't, you generally need to abort the encode, and hope that that faulty chunk gets encoded next time.

VirulentPip

12th May 2022, 12:32

Is there anything I can do to force the Encoding Client to open on a secondary monitor? Whenever a new encode starts it always re-opens on Primary Display, even when using software such as UltraMon (free trial) UltraMon works fine for RipBot (main) but not the Encoder Client or Server windows.

The reason I ask is that I have the Encoding Client on my Gaming PC as I have a very high TBW NVME drive on here and I like to have the Ripbot stuff open on a second monitor. However when a new one starts the Encoding Client will either minimise the game if in Fullscreen or appear on top if in Windowed Borderless, it doesn't appear on top if keep the Encoding Client on Primary Monitor while in Windowed Borderless.

guest

15th May 2022, 02:50

guest

15th May 2022, 05:30

I have been doing a couple of encodes today (1st time, in a LONG time), and to anyone that is using "my" build's and using ANY of the SMDegrain filters might see very slow, and low CPU usage, encodes.

So please check how you've got this setup...

https://i.imgur.com/NQoaeu9.jpg

Note:- I'm running an AMD Ryzen 5950X, hence the 32 cores setting, but I think it will "self adjust" if you're using less cores.

Before changing these, the CPU usage was VERY low, so a little trial and error might be needed to get your system working to it's maximum.

Ryushin

16th May 2022, 19:30

I have been doing a couple of encodes today (1st time, in a LONG time), and to anyone that is using "my" build's and using ANY of the SMDegrain filters might see very slow, and low CPU usage, encodes.

Hi Pauly Dunne,

I've set up your tools folder and started testing the SMDegrain over the weekend. Just wanted to add the SMDegrain needs to be added to the filter list and it works after that. I also have to turn off Auto Updates because Atak's immediately overwrites some of the files. In addition I had to upgrade my encoding servers to your build as well in order to work.

I've just started to look into the options for SMDegrain to tune it to my needs. I'm curious about the main differences in your script names. What does the "E-H" mean? I should also note that your scripts call "SMDegrain 350d.avs" which uses CUDA. Since I don't have an NVidia card, I've been modifying the scripts to use "SMDegrain 350d cpu.avs".

Right now the the tests I've run as follows:
Source: Invasion of the Body Snatchers 4K - First 15 Minutes
CQ 18
No Degraining: 7.4 GB
MDegrain2 thSAD=400: 2.9 GB
MDegrain3 thSAD=600: 2.4 GB
SMDegrain E-H std: 5.7 GB
SMDegrain E-H BM3D CPU: (too slow: .28 fps)
SMDegrain E-H KNL: (does not work, probably needs GPU)
SMDegrain Hard std: 6.3 GB

I'll dig into it further to tune SMDegrain for heavier degraining, because as of right now, plain MDegrain2 is giving me a much smaller file size.

guest

16th May 2022, 23:23

Hi Pauly Dunne,

I've set up your tools folder and started testing the SMDegrain over the weekend. Just wanted to add the SMDegrain needs to be added to the filter list and it works after that. I also have to turn off Auto Updates because Atak's immediately overwrites some of the files. In addition I had to upgrade my encoding servers to your build as well in order to work.

I've just started to look into the options for SMDegrain to tune it to my needs. I'm curious about the main differences in your script names. What does the "E-H" mean? I should also note that your scripts call "SMDegrain 350d.avs" which uses CUDA. Since I don't have an NVidia card, I've been modifying the scripts to use "SMDegrain 350d cpu.avs".

Right now the the tests I've run as follows:
Source: Invasion of the Body Snatchers 4K - First 15 Minutes
CQ 18
No Degraining: 7.4 GB
MDegrain2 thSAD=400: 2.9 GB
MDegrain3 thSAD=600: 2.4 GB
SMDegrain E-H std: 5.7 GB
SMDegrain E-H BM3D CPU: (too slow: .28 fps)
SMDegrain E-H KNL: (does not work, probably needs GPU)
SMDegrain Hard std: 6.3 GB

I'll dig into it further to tune SMDegrain for heavier degraining, because as of right now, plain MDegrain2 is giving me a much smaller file size.

Hi Ryushin, glad to know you're finally getting around to trying the build :)

"E-H" means Extra Hard !!

And yes, BM3D does use CUDA, and hence an appropriate nVidia GPU, that is why there's a CPU version of the script.

Same with the DG & KNL scripts, the latter needing OpenCL, mainly.

And also yes to turning off the Autoupdates (this was in the "instructions").

I hope your "updating" of the servers, you just replaced the Tools folder ?!?! This is because of the Avisynth version, as most of the scripts/filters need 3.7.2.

I have gone to the trouble & expense of adding nVidia GPU's to ALL my servers so that I can use ALL the filters !!

Now, unlike you (and many others) you go for smaller files size, where as I go for bitrate, so using CQ18 just doesn't interest me.

The settings I decided on a couple of days ago, are working VERY well for me atm, I am seeing good speed, and great results, mind you I am only doing 1080p right now, but I have a 4K movie queued up, so I might try your settings with that, and see what I get.

I don't think I need to add SMDegrain to the filter list, just unchecking it, and I get 100% CPU usage :) I've just tested this, and I still think that just simply unchecking that option is all that's required, when using SMDegrain (but each to their own)

So not knowing what PC's you have working, to get those "numbers" (above), but my Ryzen's are doing a great job :)

PS:- I've just started a couple of 4K tests, and the pre filter's REALLY slow the process down, but "straight" SMDegrain's seem to do well. Have I ever said how much I HATE 4K encoding, even with several Ryzen 9's grinding away :(

Dhry

17th May 2022, 17:59

Ripbot has worked perfectly for me for years. Just today I tried starting it up and immediately see a dialog:
Runtime error 217 at 00411E42

There is an entry in Event Viewer showing the following:

Faulting application name: RipBot264.exe, version: 1.26.1.0, time stamp: 0x2a425e19
Faulting module name: KERNELBASE.dll, version: 10.0.19041.1706, time stamp: 0x40a40d01
Exception code: 0x0eedfade
Fault offset: 0x0012c3a2
Faulting process id: 0x3de8
Faulting application start time: 0x01d86a0e3aa8b2ba
Faulting application path: C:\Program Files (x86)\RipBot264\RipBot264.exe
Faulting module path: C:\Windows\System32\KERNELBASE.dll
Report Id: 559a9f50-9bcf-42cc-89e0-339a3a17fd84
Faulting package full name:
Faulting package-relative application ID:

What could be causing this?
---
Edit: Rebooted and it's working now. No idea what happened, but my PC hung at the "Waiting for System Event Notification service" during shutdown and I physically had to turn it off and back on. Evidently something in the OS got fried.

guest

18th May 2022, 05:53

So I upgraded my server using a used Supermicro motherboard and I ordered two Xeon E5-2680 V4 (14 cores) processors for it from Ebay. Got everything installed on Tuesday.

I configured my virtual machine with to use 24 cores and two threads for a total of 48 CPU threads. Gave the VM 32GB of RAM as well. I am not using NUMA.

Run the x265 Benchmark and received a score of 59.2 so I was happy with this.

Received Moonfall in 4K from Amazon and I proceeded to add it to RB. Noticed i would have to do a bit of degraining and enabled MDegrain2 with a THSAD value of 200. Started the encoding with two servers and was amazed at how slow the encoding went and how it was not utilizing the CPU.

So I did a test encoding with no MDegrain2 and it was fast. So most likely it was the threading problem with MDegrain2. My "Use multiple processing threads" setting was set to the default of 0 which I think means use a thread for each CPU thread which would be 48.

So I ran some tests by lowering the number of threads used for MDegrain2 and the results were dramatic:

1 Minute Chunk Size, First Chunk, 1 Encoding Server
No MDegrain - 6.38 fps (CPU running about 45-85%)
MDegrain2 48 threads (0 for unlimited) - 1.27 fps
MDegrain2 24 threads - 1.29 fps
MDegrain2 18 threads - 2.31 fps
MDegrain2 15 threads - 4.36 fps
MDegrain2 14 threads - 4.44 fps
MDegrain2 13 threads - 4.38 fps
MDegrain2 12 threads - 4.15 fps
MDegrain2 10 threads - 3.62 fps

1 Minute Chunk Size, First and Second Chunk, 2 Encoding servers
No MDegrain - 4.17 and 12.13 fps = 16.30 fps (CPU running about 92-100%)
MDegrain2 15 threads - 3.05 and 3.62 fps = 6.67 fps
MDegrain2 14 threads - 3.18 and 4.22 fps = 7.40 fps
MDegrain2 13 threads - 3.26 and 3.96 fps = 7.22 fps

So I hope this helps someone who has a highly threaded setup and that you will probably have to tune the MDegrain2 thread settings for each machine.

Hi Ryushin, This is all very interesting, as I seem to be having a similar problem, not a particular filter but EVERYTHING :(

I've had a dual Xeon E5-2697v2 (12 cores) setup for a while, and it's worked good 'til now, I "transplanted" it into a case, instead of being directly screwed to a table !!

The only thing I changed was the type & amount of RAM. (more)

I did a test run with it today, and it was SLOW, an old dual Xeon X5690 (a measly 6 cores) system was twice as fast, which is pretty sad.

I've tried a few "thread" settings, but nothing seems to help.

I'm running Windows Server 2022, might need to try "normal" Windows.

Ryushin

18th May 2022, 12:55

Hi Ryushin, This is all very interesting, as I seem to be having a similar problem, not a particular filter but EVERYTHING :(

I've had a dual Xeon E5-2697v2 (12 cores) setup for a while, and it's worked good 'til now, I "transplanted" it into a case, instead of being directly screwed to a table !!

The only thing I changed was the type & amount of RAM. (more)

I did a test run with it today, and it was SLOW, an old dual Xeon X5690 (a measly 6 cores) system was twice as fast, which is pretty sad.

I've tried a few "thread" settings, but nothing seems to help.

I'm running Windows Server 2022, might need to try "normal" Windows.

Did you turn off the option "Limit to the following filters only"? Some functions don't seem to thread well and cause the encode to slow down. This is why I added the SMDegrain to that list.

Is the RAM running at a slower speed, that could cause a bit of a difference as well.

Once I tuned the threading settings I was getting good performance. But that area where performance was good was pretty narrow. You're probably going to have to do some benchmarking with different thread settings and see where your sweet spot is. Perhaps NUMA is enabled as well. I was testing NUMA on my older Ryzen, I found non NUMA was better for me.

guest

19th May 2022, 08:48

Did you turn off the option "Limit to the following filters only"? Some functions don't seem to thread well and cause the encode to slow down. This is why I added the SMDegrain to that list.

Is the RAM running at a slower speed, that could cause a bit of a difference as well.

Once I tuned the threading settings I was getting good performance. But that area where performance was good was pretty narrow. You're probably going to have to do some benchmarking with different thread settings and see where your sweet spot is. Perhaps NUMA is enabled as well. I was testing NUMA on my older Ryzen, I found non NUMA was better for me.

Hi, I am not using this PC as the Client, so the ALL the settings are dependent on the the settings of the Client (main) PC, so changing them is irrelevant.

I'm still not convinced that SMDegrain needs to added to the Threads list !!!

The RAM is the same speed, 1066, it was what came with the Dell R720's, I didn't need 128Gb in them, so I borrowed 64Gb.

I'm doubtful that it's a Windows Server 2022 issue....

Correct me if I'm wrong, but I don't think NUMA is an option with Ryzen CPU's, it's called something else, but I don't think it works the same way, but I'm running Intel Xeon's, and have the NUMA function disabled.

But one very annoying error is randomly happening AGAIN, x265 [error]: unable to open input file, and with x264, as well :(

I did my 1st encode on one of the R720's, and initially got this error, but then I realised I hadn't updated vc-redist(s), after that it worked, but was a bit slow & stalled a few times, but it was runnung @ 100% CPU usage, 'til it stalled, but it completed the chunk, and then with the next encode the error returned, so I turned it off in disgust !!!!

Ryushin

20th May 2022, 13:13

Hi, I am not using this PC as the Client, so the ALL the settings are dependent on the the settings of the Client (main) PC, so changing them is irrelevant.

I'm still not convinced that SMDegrain needs to added to the Threads list !!!

The RAM is the same speed, 1066, it was what came with the Dell R720's, I didn't need 128Gb in them, so I borrowed 64Gb.

I'm doubtful that it's a Windows Server 2022 issue....

Correct me if I'm wrong, but I don't think NUMA is an option with Ryzen CPU's, it's called something else, but I don't think it works the same way, but I'm running Intel Xeon's, and have the NUMA function disabled.

But one very annoying error is randomly happening AGAIN, x265 [error]: unable to open input file, and with x264, as well :(

I did my 1st encode on one of the R720's, and initially got this error, but then I realised I hadn't updated vc-redist(s), after that it worked, but was a bit slow & stalled a few times, but it was runnung @ 100% CPU usage, 'til it stalled, but it completed the chunk, and then with the next encode the error returned, so I turned it off in disgust !!!!

When I was testing MDegrain2 when it was just first being added as an option in RB it was painfully slow. Atak added the threading option for certain functions that would work with threading while other functions would not.

I think NUMA is NUMA. I just built a 24 core EPYC server and Linux reports NUMA is available.

I noticed that I got the unable to open input file on certain scripts in Pauly Dunne's tools. The SMDegrain CPU is working fine for me and that is what I'm doing testing with. I'm seeing 7-12 FPS with 4K with SMDegrain depending on options.

Ryushin

20th May 2022, 21:20

So I've spent a few days getting to know SMDegrain and it's options compared to MDegrain. At first using Pauly Dunne's custom scripts for SMDegrain did not provide better results than MDegrain and were in fact worse size wise than just using MDegrain. Pauly encodes for bit rate and I prefer Constant Quality since the file size actually tells me how clean the output is.

I found good documentation for SMDegrain here: http://avisynth.nl/index.php/SMDegrain

Originally I was testing with a 15 minute clip from Invasion of the Body Snatchers in 4K. But the better test case was to use Blade Runner in 4K since the 15 minutes clip from the beginning has lots of grain, haze, cigarette smoke, etc, plus it will show off color banding if there are problems.

My test results:
Source: Blade Runner 1982 4K - 15 Minutes cut from 2:58-17:58
Source Size: 4.0 GB
Constant Quality 18
6.1 GiB: No Degraining: (Note, file size is larger than the source because of so much grain)
2.2 GiB: MDegrain2 thSAD=400
1.6 GiB: MDegrain3 thSAD=600
Note: I Find MDegrain3 can cause problems with example of a close up a face, you can see the facial texture, but if they suddenly turn their face, it blurs, then you can see the texture again when the turn finishes. Very annoying to watch and as such, I only used MDegrain3 on the absolute grainiest sources.
4.1 GiB: SMDegrain(video) (no options) (See docs for the default options)
2.8 GiB: SMDegrain(video,tr=2,thSAD=400,thSCD2=80)
2.8 GiB: SMDegrain(video,tr=2,thSAD=400,thSADC=200,thSCD2=80)
1.7 GiB: SMDegrain(video,tr=2,thSAD=800,thSADC=400,thSCD2=80)
1.9 GiB: SMDegrain(video,tr=2,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true)
1.0 GiB: SMDegrain(video,tr=4,thSAD=800,thSADC=400,thSCD2=80)
1.1 GiB: SMDegrain(video,tr=4,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true)
828 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80)
3.1 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=-1,refinemotion=true)
1.2 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=0,refinemotion=true)
967 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=1,refinemotion=true)
767 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true)

So increasing the "tr" value is essentially which MDegrain you want to use. In the case above, from 2-8. The file size was actually quite good using a tr of 8.

The contrasharp provides sharpening. I found adding the contrasharp provided a nice detail enhancement but it really needed to be paired with using a prefilter which blurs out the grain a bit before sharpening. A prefilter of -1 provides no filtering, 0 is light, 1 is medium, and 2 is strong. Blocky sources need to use 3 for the prefilter. Refinemotion helps with detecting motion, but if the prefilter is too strong it can cause problems.

I watched the output of the 767 MiB file this morning and compared it to the source and the difference is outstanding.

Atak, please add SMDegrain to RipBot264. It seems to be a swiss army knife of sorts. You will probably need to give perhaps three presets and hopefully and Advanced that lets you tune some of the parameters such as tr, thSAD, contrasharp, prefilter, refinemotion, etc. I almost think SMDegain could be so good you can retire MDegrain at this point.

You can download the 15 minute Blade Runner source file and the 767 MiB file to see the differences from here:
https://cloud.chrisdos.com/s/mdMDjgQMbMCdaes The link expires in two weeks.

I've ripped some of my grainiest 4K movies to try things out on and watch them and see how it fairs.

Pauly Dunne, thank you for the hard work of providing the SMDegrain scripts and tools. I'll try and take SMDegrain pieces and integrate them into to RB's default tools folder so I can turn on auto updates again.

guest

21st May 2022, 03:03

So I've spent a few days getting to know SMDegrain and it's options compared to MDegrain. At first using Pauly Dunne's custom scripts for SMDegrain did not provide better results than MDegrain and were in fact worse size wise than just using MDegrain. Pauly encodes for bit rate and I prefer Constant Quality since the file size actually tells me how clean the output is.

I found good documentation for SMDegrain here: http://avisynth.nl/index.php/SMDegrain

Atak, please add SMDegrain to RipBot264. It seems to be a swiss army knife of sorts. You will probably need to give perhaps three presets and hopefully and Advanced that lets you tune some of the parameters such as tr, thSAD, contrasharp, prefilter, refinemotion, etc. I almost think SMDegain could be so good you can retire MDegrain at this point.

Pauly Dunne, thank you for the hard work of providing the SMDegrain scripts and tools. I'll try and take SMDegrain pieces and integrate them into to RB's default tools folder so I can turn on auto updates again.

Hi Ryushin,

WOW, you HAVE been busy, well done !!!

I am SO SO happy that you have pursued this, I knew that you would NOT be disappointed with SMDegrain, once you got your head around it.

So I have compared & cross referenced some of your "scripts" with "my" filters names, and it seems that you are mainly using the E-H (extra hard) settings, and just changing the "tr" values.

There are so many others to choose from with lower settings, already.

For example :-

3.1 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=-1,refinemotion=true) is basically the SMDegrain E-H Minblur#-1 filter, and so on...

1.2 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=0,refinemotion=true) SMDegrain E-H Minblur#0

967 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=1,refinemotion=true) SMDegrain E-H Minblur#1

767 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true) SMDegrain E-H Minblur#2

So if your starting from scratch with your own SMDegrain scripts, they are already there !!!, you just need to look in the Custom folder.

Should I rename them to something a little more explanatory ??

Now I know you're no stranger to modifying scripts, so you can go crazy modding any of these to suit your needs, and just save them in the Custom folder, and delete any that you don't want.

So I will take this opportunity to provide some basic "hardware" requirements for some of the SMDegrain variants :-

Any BM3D CPU filters requires a CPU with AVX2 instructions, so that means basically most hi-end Ryzens, Threadrippers, etc, but very few Intel's, maybe Gen 12.

I would suggest you check the specs of the CPU's you use !!

Any BM3D CUDA filters requires an nVidia GPU (faster the better).

Any DGDecNV filters requires an nVidia GPU for the nvenc/nvdec, functions.

Any KNLMeansCL filters requires OpenCL, either nVidia or AMD GPU's.

Most of the other filters are good to go.

Also, if you want to suss out more "Dogway" modded SMDegrain stuff, go here :-

https://forum.doom9.org/showthread.php?t=182881

The link you posted for SMDegrain info is now a little dated, due to Dogway's mod's. (that's what I use)

And don't ignore some of the other filters in the Custom list...

I would be shocked if Atak did do anything to implement some SMDegrain filters, but never, say never.

And as for Auto Updates, just disable it, to avoid problems !!!!

Cheers & thanks :)

PS:- I have downloaded your Bladerunner sample, will take a close look.

guest

21st May 2022, 03:19

When I was testing MDegrain2 when it was just first being added as an option in RB it was painfully slow. Atak added the threading option for certain functions that would work with threading while other functions would not.

I think NUMA is NUMA. I just built a 24 core EPYC server and Linux reports NUMA is available.

I noticed that I got the unable to open input file on certain scripts in Pauly Dunne's tools. The SMDegrain CPU is working fine for me and that is what I'm doing testing with. I'm seeing 7-12 FPS with 4K with SMDegrain depending on options.

Hello again,

I did some more testing on my "slow" dual E5-2697v2 yesterday, and tried all sorts of setting's to see if I could get some performance back.

And I ended up changing a couple of x265 commands, and filter settings.

I even tried NUMA & no NUMA, and with NUMA enabled, made a very noticeable speed increase, and if you only use 2 encoding port address's, you have one for /node 0, and one for /node 1...nice.

Speaking on NUMA, Ryzens don't have it, BUT Threadripper's & EPYC's, apparently do....

And I still don't think SMDegrain needs to be added to the filter thread list, I actually noticed a slight increase in speed, with it not even IN the list, in the RipBot264.ini file.

And I think I had a bit of a brain fart with the R720 the other day when I got that x265 error....as it had previously worked ok on an encode, but the next encode, I got the error...turns out I was using a different filter which required AVX2, which the Xeon's don't have.....

I'm seeing 7-12 FPS with 4K with SMDegrain depending on options. So what machine(s) are you running to get this speed ??, and how many DE server ports are you using ??

guest

21st May 2022, 03:28

You can download the 15 minute Blade Runner source file and the 767 MiB file to see the differences from here:
https://cloud.chrisdos.com/s/mdMDjgQMbMCdaes The link expires in two weeks.

I just had a look at your sample, and even on my laptop, it IS impressive, that's for sure :)

You just need to convert the audio to E-AC3 @ 1536kbps.

NICE, cheers.

Ryushin

21st May 2022, 12:08

I just had a look at your sample, and even on my laptop, it IS impressive, that's for sure :)

You just need to convert the audio to E-AC3 @ 1536kbps.

NICE, cheers.

Oh, the audio for the test clip was just minimal as I really did not want the audio size to affect my testing. The finished files have AC3 Core and TrueHD+Atmos in the files or DTS (compressed) and DTS:X if available.

Ryushin

21st May 2022, 12:14

So what machine(s) are you running to get this speed ??, and how many DE server ports are you using ??

My main RP machine is a virtual machine with 24 cores and 48 threads. The Linux KVM Hypervisor has dual E5-2680 v4 @ 2.40GHz (14 cores) and passes 24 cores and 48 threads to the VM. I have a first gen Ryzen 1800X that I use for an encoding server. With those I see 7-12 FPS with 4K just using SMDegrain CPU and no GPU assistance.

Ryushin

21st May 2022, 18:45

So I encoded Invasion of the Body Snatchers 4K over last night and I started watching it this morning and I encountered a problem in one of the scenes in the movie.

It's a dark scene with two people talking and their faces are in the light. If you watch the women's cheek you will see that seems degrain for a quarter second, then not degrain, and back and forth. It was very odd.

I extracted that 90 second scene out and looked it it in detail and the source has some odd grain around the women's cheek and that is probably where the problem occurs. I ran a bunch of tests using different options against the sample in order to find out how to solve the problem. Reducing the block size from the default of 16 to 8 solved the problem. As a bonus the file size was 11% smaller than before using the default block size. Encoding takes a hit since I'm essentially quadrupling the number of blocks to analyze. Encoding was about 30-40% slower.

Link to the sample source and the two encoded files (link valid for two weeks):
https://cloud.chrisdos.com/s/6JkTfF2etSo8Ln8

I've changed my scripts to use a block size of 8 now:
SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true,blksize=8)

Pauly Dunne: You have a lot of scripts. Most would not run on my system since I don't have a GPU. So you may want to add a GPU/CPU in the name or provide a different folders with GPU and CPU optimized scripts. You may also want to condense the scripts into an advanced vs simplified. Perhaps just have 4-5 scripts from very light to very hard and then all the other scripts in a different folder for those that want to try all the different options and alternatives to SMDegrain.

guest

22nd May 2022, 03:23

Pauly Dunne: You have a lot of scripts. Most would not run on my system since I don't have a GPU. So you may want to add a GPU/CPU in the name or provide a different folders with GPU and CPU optimized scripts. You may also want to condense the scripts into an advanced vs simplified. Perhaps just have 4-5 scripts from very light to very hard and then all the other scripts in a different folder for those that want to try all the different options and alternatives to SMDegrain.

Hello again,

You're like a dog with a bone....VERY particular (which is good)

Yes, I realise there are a LOT of scripts, and I doubt that many of them would get used, once the user has "favourite's", then they could quite easily copy the others to another folder, and greatly reduce the drop down list.

As for using a different folder, for different type or category's, how would you "call them up" from within RipBot ???

The only way I've known is to have EVERYTHING in the Custom folder, so it is available with the "Custom Script" option "box".

If you create a New Folder within the Custom Folder, it does not show up in the Custom Script list !!!!

So if you know a way to do that, please let me know, and I will add it !!!

Same goes for Advanced & Simplified, how ???

However, I do like your suggestion of using CPU or GPU in the scripts name :)

Anyway, it's a good base for users to start with, and customise to meet their own requirements, and remove the rest.

I have downloaded your 2 sample's, and I will have to run them on a PC, but I DID noticed they are a very high bitrate, coming from you (I know they are only tests), but the Blade Runner sample @ CQ18, was around 6700kb/s, and todays were in the low to mid 40mb/s.

So keep up the good work, and if you have any suggestions / changes or additions, please let me know.....Upward & Onward :)

Cheers & thanks.

guest

22nd May 2022, 10:58

SMDegrain E-H KNL: (does not work, probably needs GPU)

I stumbled across some new/old info about OpenCL support for Intel CPU's that maybe you could suss out (I know I'm going to try again).

Atak posted this on another Forum :-

https://forum.doom9.org/showthread.php?p=1969319#post1969319

So I downloaded the Intel OpenCL Driver (needs an Intel account,tho).

So if you wanted to try a KNLMeansCL script without a GPU, you could mod the script to use device_type="CPU".

Ryushin

22nd May 2022, 12:27

I have downloaded your 2 sample's, and I will have to run them on a PC, but I DID noticed they are a very high bitrate, coming from you (I know they are only tests), but the Blade Runner sample @ CQ18, was around 6700kb/s, and todays were in the low to mid 40mb/s.

This is why I like Constant Quality. I'm not wasting bits if they are not needed. Even after degraining there was still some grain and as such CQ raised the bit rate to make sure the quality of the entire scene was the same as the whole clip. Another example is Avengers Endgame 4K is so clean the entire 3 hour movie comes out to 5.76 GiB (4553 kb/s bit rate) without any degraining whatsoever using CQ18. ideally, if you watched Endgame with a CQ18 and x265, it should be visually indistinguishable to an encode with hard bit rate of 90 Mb/s.

Using CQ also allows me to judge how clean the output is. I had another 4K movie that looked pretty clean and just had some light grain and it came out to be 18GB in size using CQ18 with no degraining. Just adding a little MDegrain2 with a thSAD=100 brought the file size down to 8GB still with CQ18. There really is no downside to using constant quality. Only reason I see to use a hard bit rate is if I'm trying to fit it on a medium that has a finite size limitation.

Ryushin

22nd May 2022, 12:31

So if you wanted to try a KNLMeansCL script without a GPU, you could mod the script to use device_type="CPU".

I think SMDegrain gives me the all the options I want at this point. The prefilter options and sharpening that actually works instead of adding artifacts like my tests gave me before. I'm very happy with SMDegrain at this point and I'll be creating scripts using it almost exclusively.

guest

23rd May 2022, 03:33

Yes, I realise there are a LOT of scripts, and I doubt that many of them would get used, once the user has "favourite's", then they could quite easily copy the others to another folder, and greatly reduce the drop down list.

As for using a different folder, for different type or category's, how would you "call them up" from within RipBot ???

The only way I've known is to have EVERYTHING in the Custom folder, so it is available with the "Custom Script" option "box".

If you create a New Folder within the Custom Folder, it does not show up in the Custom Script list !!!!

So if you know a way to do that, please let me know, and I will add it !!!

Same goes for Advanced & Simplified, how ???

So keep up the good work, and if you have any suggestions / changes or additions, please let me know.....Upward & Onward :)

Cheers & thanks.

Hi Ryushin,

Hi, can you please comment on this...

Thanks

Ryushin

23rd May 2022, 10:44

Hi Ryushin,

Hi, can you please comment on this...

Thanks

Since you can't have subfolders and such, my suggestions would be something like the following:

In the custom scripts, have only about a dozen scripts with generic names, such as:
SMDegrain_GPU_Light/Medium/Hard/Very_Hard
SMDegrain_CPU_Light/Medium/Hard/Very_Hard

Then in your install instructions, provide another folder with the advanced scripts and tell them them where to place them. I would also add the requirements in the script name, such as GPU, CPU, AVX2, etc.

Then the install instructions can also have an advanced section and you can lay out what HQD3D (which I think can only do 8-bit color), CQTMGC, BM3D, etc, can do. Maybe even provide a link that has more details such as that link I provided for SMDegrain.

In the beginning, all the scripts will overwhelm someone. There is also probably quite a bit of overlap. Such as is using BM3D going to give better results than just SMDegrain, and if so, what are the advantages/disadvantages.

Ryushin

23rd May 2022, 12:53

So I encoded Invasion of the Body Snatchers 4K over last night and I started watching it this morning and I encountered a problem in one of the scenes in the movie.

It's a dark scene with two people talking and their faces are in the light. If you watch the women's cheek you will see that seems degrain for a quarter second, then not degrain, and back and forth. It was very odd.

I extracted that 90 second scene out and looked it it in detail and the source has some odd grain around the women's cheek and that is probably where the problem occurs. I ran a bunch of tests using different options against the sample in order to find out how to solve the problem. Reducing the block size from the default of 16 to 8 solved the problem. As a bonus the file size was 11% smaller than before using the default block size. Encoding takes a hit since I'm essentially quadrupling the number of blocks to analyze. Encoding was about 30-40% slower.

Link to the sample source and the two encoded files (link valid for two weeks):
https://cloud.chrisdos.com/s/6JkTfF2etSo8Ln8

I've changed my scripts to use a block size of 8 now:
SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true,blksize=8)

So the encoding performance hit was pretty significant probably closer to 50% slower, so I did some more testing. Removing the thSCD2=80 option (which changes it back to it's default of 130) fixed the problem as well. I know Ripmann changed the option because the scene change detection was not being detected properly and it was merging frames that were not related to each other. Perhaps since SMDegrain brightens to the scene to find more grain before degraining the scene detection will work better. I guess we'll see what happens as I'll go back to using the default of thSCD2=130 by leaving the option out.

guest

24th May 2022, 07:43

Since you can't have subfolders and such, my suggestions would be something like the following:

In the custom scripts, have only about a dozen scripts with generic names, such as:
SMDegrain_GPU_Light/Medium/Hard/Very_Hard
SMDegrain_CPU_Light/Medium/Hard/Very_Hard

Then in your install instructions, provide another folder with the advanced scripts and tell them them where to place them. I would also add the requirements in the script name, such as GPU, CPU, AVX2, etc.

Then the install instructions can also have an advanced section and you can lay out what HQD3D (which I think can only do 8-bit color), CQTMGC, BM3D, etc, can do. Maybe even provide a link that has more details such as that link I provided for SMDegrain.

In the beginning, all the scripts will overwhelm someone. There is also probably quite a bit of overlap. Such as is using BM3D going to give better results than just SMDegrain, and if so, what are the advantages/disadvantages.

Hi, thanks for that...that IS a lot of work, and again, there are so many variants of all the pre-filters, I wouldn't know where to start, and where to put what....maybe a bit re-naming & explanation might be enough, and the user can sort it out to their liking.

I know there's a LOT that I personally wouldn't use.

I'll have a good look at it, when I get the chance...maybe I'll get a "lightbulb" moment :)

guest

24th May 2022, 08:08

So the encoding performance hit was pretty significant probably closer to 50% slower, so I did some more testing. Removing the thSCD2=80 option (which changes it back to it's default of 130) fixed the problem as well. I know Ripmann changed the option because the scene change detection was not being detected properly and it was merging frames that were not related to each other. Perhaps since SMDegrain brightens to the scene to find more grain before degraining the scene detection will work better. I guess we'll see what happens as I'll go back to using the default of thSCD2=130 by leaving the option out.

So I had a look at those clips you posted, and you must have a very good eye for "defect's", I don't think I could pick it...but it lead you to modding the script until it's fixed...

So, I can't help but notice you're are mainly using the Extra Hard (might have a name change soon), scripts, and only changing the "tr=" value, and maybe the pre-filter #.

With the Light, Medium & Hard scripts the "tr=" value is changed, along with a lot of the "th" type values, have you tried any of them for lesser noisy sources ??

Also, I had another good look at the SMDegrain link's info, and it would be interesting to try this script :-

I've changed my scripts to use a block size of 8 now:
SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true,blksize=8)

SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD1=300,thSCD2=130,contrasharp=true,prefilter=2,refinemotion=true,blksize=8)

I think thSCD1 works in conjunction with blksize values.

And you could leave thSCD2 out (like you mentioned)

I wonder if there's some way to incorporate a 2 pass SMDegrain run, like here (and the post below):-

https://forum.doom9.org/showthread.php?p=1969268#post1969268

Anyway, regardless to all that, I'm just so happy that you have finally come around to using & preferring SMDegrain, it was one aim of mine, when I started doing all these additions, so thankyou :)

And please, if you come up with some different scripts or filters, don't forget to let me know, either here on the Forum, a PM, or email.

PS:- It would be interesting to try these out on the Jetson's, and see what result you get, even though you're happy with how you did them, several months back.

Ryushin

24th May 2022, 11:45

So, I can't help but notice you're are mainly using the Extra Hard (might have a name change soon), scripts, and only changing the "tr=" value, and maybe the pre-filter #.

With the Light, Medium & Hard scripts the "tr=" value is changed, along with a lot of the "th" type values, have you tried any of them for lesser noisy sources ??

Anyway, regardless to all that, I'm just so happy that you have finally come around to using & preferring SMDegrain, it was one aim of mine, when I started doing all these additions, so thank you :)

PS:- It would be interesting to try these out on the Jetson's, and see what result you get, even though you're happy with how you did them, several months back.

I had been keeping Invasion of the Body Snatches ripped to do testing against SMDegrain, just needed the time and motivation to do it. For 95% of everything, MDegrain2 did a fantastic job and sometimes if it "works" just leave it alone. I appreciate your work to bring SMDegrain and other tools to RipBot. Now if only Atak would integrate SMDegrain into it and it would be nice to hear from him more often.

But those super grainy sources always bothered me and I did not like using MDegrain3 because of the blurring I saw in skin textures when there was movement. SMDegrain seems to do really well and the results are natural looking.

I have not tested any lighter grainy sources yet, but I feel it will at least do as well as MDegrain2. I probably won't be doing any blurring/sharpening on the lighter sources. I'll start with the tr=2 for lighter degrain and tr=4 probably for medium degrain. Strangely, the tr=2 would be the equivalent of MDegrain2 and tr=8 would be MDegrain8. But the file size difference between tr=2 and tr=8 was dramatic and I saw no blurring issues like when just using MDegrain3. Encoding performance is still good using tr=8. I tried tr=12 trying to solve the issue I mentioned but the slow down was not worth it.

I have Superman The Animated Series still up on the server. I have a run with MDegrain2 and using Handbrake with NLMeans animation Tune. I'll run SMDegrain against it as well to see where it ends up and compare the three. Jetsons and Flintstones I'll probably just leave alone as the results from Handbrake are stunning, just the encoding was so slow it was just painful to do.

slalom

26th May 2022, 17:00

The reason I ask is that I have the Encoding Client on my Gaming PC as I have a very high TBW NVME drive on here and I like to have the Ripbot stuff open on a second monitor. However when a new one starts the Encoding Client will either minimise the game if in Fullscreen or appear on top if in Windowed Borderless, it doesn't appear on top if keep the Encoding Client on Primary Monitor while in Windowed Borderless.
keep it minimized while gaming
Hello again,

I did some more testing on my "slow" dual E5-2697v2 yesterday, and tried all sorts of setting's to see if I could get some performance back.

And I ended up changing a couple of x265 commands, and filter settings.

I even tried NUMA & no NUMA, and with NUMA enabled, made a very noticeable speed increase, and if you only use 2 encoding port address's, you have one for /node 0, and one for /node 1...nice.
That's what it's for, for 2 CPUs

guest

27th May 2022, 01:30

That's what it's for, for 2 CPUs

Hi slalom, you haven't posted in quite a while...

I kind of agree, but it's not always the best option.

Depending on the workload.

Axelstrife

27th May 2022, 17:46

Atak_Snajpera

27th May 2022, 21:59

Record your desktop and upload your video somewhere. This way I will see what you see and help me to identify your issue.

slalom

27th May 2022, 23:24

Hi slalom, you haven't posted in quite a while...

I kind of agree, but it's not always the best option.

Depending on the workload.
I am around... :cool:

guest

28th May 2022, 01:52

Hi, Is there there a guide of some sort for the distributed encoding? i'm trying to figure why it isn't working for me.

Ive got encoding servers open on both PC's and both PC's Ip's entered into the encoding client on the main PC both say online, but when i add a movie to encode and hit start it just says "please wait" and then after a minute or two says "All jobs finished" pop up notification despite nothing actually happening.

Ive got both PC's windows firewall disabled to eliminate that as the issue.
I have read about some software causing issues which is why i reinstalled windows on one PC and the other received a fresh reinstall about three days ago and only has 6 programs installed since then (Plex, Sonarr, Radarr, SABnzbd, media info, 7zip)

Is there a port that needs to be open? ive got no idea why this isnt working

Hi, you posted a very similar "question" nearly 3 years ago...you were having problems with DE....did you fix it, back then ??

You've listed the programs you've installed, BUT have you got an nVidia GPU, as the drivers (or part there of) can cause a lot of problems.

And apparently some Logitech stuff, does to.

Maybe you need to go back to your 1st post, and see what replies you got....their might be a "fix" mentioned back then.

As long as the other PC(s) have slightly different IP address's, and on the same group, it should work.

It's probably something simple, but it's frustrating !!!!

If you can't get it to work, do what Atak suggested, and he WILL be able to help, I'm sure.

Axelstrife

28th May 2022, 07:02

Record your desktop and upload your video somewhere. This way I will see what you see and help me to identify your issue.
Here you go, i am recording from Gaming PC and remoting into my plex server PC.
https://youtu.be/GJAZRLf6dzA

Axelstrife

28th May 2022, 07:13

Hi, you posted a very similar "question" nearly 3 years ago...you were having problems with DE....did you fix it, back then ??

You've listed the programs you've installed, BUT have you got an nVidia GPU, as the drivers (or part there of) can cause a lot of problems.

And apparently some Logitech stuff, does to.

Maybe you need to go back to your 1st post, and see what replies you got....their might be a "fix" mentioned back then.

As long as the other PC(s) have slightly different IP address's, and on the same group, it should work.

It's probably something simple, but it's frustrating !!!!

If you can't get it to work, do what Atak suggested, and he WILL be able to help, I'm sure.

The issue back then was different, i actually did have that issue this time but i solved it and then ran into this issue.

both PC's have Nvidia GPU's RTX 3090 and GT 720(9900kf in this PC doesn't have igpu) ive not installed any drivers from Nvidia's website yet so it's only whatever drivers windows uses.
No logitech software is installed

I posted a Youtube link the a screen recording showing exactly what happens in reply to Atak.

guest

28th May 2022, 07:39

Here you go, i am recording from Gaming PC and remoting into my plex server PC.
https://youtu.be/GJAZRLf6dzA

Hi,

I have just watched this clip, and unfortunately, it's not really clear enough to see exactly what's going on. (It helped a little when I put my glasses on).

You might want to change the UAC settings, so you don't get that popup warning.

What happens if you don't use DE, just the single basic encode ???

Are you using the Pauly Dunne build ???

I use that, and don't have any issues...

Do you have any other PC's you can connect to ??

Axelstrife

28th May 2022, 08:16

Hi,

I have just watched this clip, and unfortunately, it's not really clear enough to see exactly what's going on. (It helped a little when I put my glasses on).

You might want to change the UAC settings, so you don't get that popup warning.

What happens if you don't use DE, just the single basic encode ???

Are you using the Pauly Dunne build ???

I use that, and don't have any issues...

Do you have any other PC's you can connect to ??

THANKYOU!, Your comment just lead me to figure out why it wasn't working.

I was manually opening "encoding client" not knowing it would automatically open when i start an encode which i just discovered when attempting to encode only on a single PC but i didn't turn off distributed encoding in the settings so the encoding client popped up.

Thanks for the UAC tip, those are annoying.