Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
10th May 2022, 15:14 | #19461 | Link | |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
Quote:
Code:
EncodingServer.exe /restart-if-no-progress
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
|
11th May 2022, 00:47 | #19462 | Link | |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
Quote:
I use to encode overnight, back in the day, but nearly every time there would be a failure, while I was asleep, so I didn't bother doing that anymore, just wasn't worth it... So thanks for the great feedback.. Cheers
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) |
|
11th May 2022, 00:51 | #19463 | Link |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
Yeah, I'm afraid to say that this doesn't always work...I have been using this command for a long time, and if a chunk doesn't want to co-operate, it won't, you generally need to abort the encode, and hope that that faulty chunk gets encoded next time.
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) |
12th May 2022, 12:32 | #19464 | Link |
Registered User
Join Date: Apr 2020
Posts: 16
|
Is there anything I can do to force the Encoding Client to open on a secondary monitor? Whenever a new encode starts it always re-opens on Primary Display, even when using software such as UltraMon (free trial) UltraMon works fine for RipBot (main) but not the Encoder Client or Server windows.
The reason I ask is that I have the Encoding Client on my Gaming PC as I have a very high TBW NVME drive on here and I like to have the Ripbot stuff open on a second monitor. However when a new one starts the Encoding Client will either minimise the game if in Fullscreen or appear on top if in Windowed Borderless, it doesn't appear on top if keep the Encoding Client on Primary Monitor while in Windowed Borderless. |
15th May 2022, 02:50 | #19465 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 1,003
|
Quote:
Maybe Multimon trial version hasn't got all it's features enabled.. Have you heard of, or tried Display Fusion Pro ?? Good luck.. (sent you a PM) Last edited by TDS; 15th May 2022 at 03:05. |
|
15th May 2022, 05:30 | #19466 | Link |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
Thread settings..
I have been doing a couple of encodes today (1st time, in a LONG time), and to anyone that is using "my" build's and using ANY of the SMDegrain filters might see very slow, and low CPU usage, encodes.
So please check how you've got this setup... https://i.imgur.com/NQoaeu9.jpg Note:- I'm running an AMD Ryzen 5950X, hence the 32 cores setting, but I think it will "self adjust" if you're using less cores. Before changing these, the CPU usage was VERY low, so a little trial and error might be needed to get your system working to it's maximum.
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) |
16th May 2022, 19:30 | #19467 | Link | |
Registered User
Join Date: Mar 2011
Posts: 433
|
Quote:
I've set up your tools folder and started testing the SMDegrain over the weekend. Just wanted to add the SMDegrain needs to be added to the filter list and it works after that. I also have to turn off Auto Updates because Atak's immediately overwrites some of the files. In addition I had to upgrade my encoding servers to your build as well in order to work. I've just started to look into the options for SMDegrain to tune it to my needs. I'm curious about the main differences in your script names. What does the "E-H" mean? I should also note that your scripts call "SMDegrain 350d.avs" which uses CUDA. Since I don't have an NVidia card, I've been modifying the scripts to use "SMDegrain 350d cpu.avs". Right now the the tests I've run as follows: Source: Invasion of the Body Snatchers 4K - First 15 Minutes CQ 18 No Degraining: 7.4 GB MDegrain2 thSAD=400: 2.9 GB MDegrain3 thSAD=600: 2.4 GB SMDegrain E-H std: 5.7 GB SMDegrain E-H BM3D CPU: (too slow: .28 fps) SMDegrain E-H KNL: (does not work, probably needs GPU) SMDegrain Hard std: 6.3 GB I'll dig into it further to tune SMDegrain for heavier degraining, because as of right now, plain MDegrain2 is giving me a much smaller file size. Last edited by Ryushin; 16th May 2022 at 19:34. |
|
16th May 2022, 23:23 | #19468 | Link | |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
Quote:
"E-H" means Extra Hard !! And yes, BM3D does use CUDA, and hence an appropriate nVidia GPU, that is why there's a CPU version of the script. Same with the DG & KNL scripts, the latter needing OpenCL, mainly. And also yes to turning off the Autoupdates (this was in the "instructions"). I hope your "updating" of the servers, you just replaced the Tools folder ?!?! This is because of the Avisynth version, as most of the scripts/filters need 3.7.2. I have gone to the trouble & expense of adding nVidia GPU's to ALL my servers so that I can use ALL the filters !! Now, unlike you (and many others) you go for smaller files size, where as I go for bitrate, so using CQ18 just doesn't interest me. The settings I decided on a couple of days ago, are working VERY well for me atm, I am seeing good speed, and great results, mind you I am only doing 1080p right now, but I have a 4K movie queued up, so I might try your settings with that, and see what I get. I don't think I need to add SMDegrain to the filter list, just unchecking it, and I get 100% CPU usage I've just tested this, and I still think that just simply unchecking that option is all that's required, when using SMDegrain (but each to their own) So not knowing what PC's you have working, to get those "numbers" (above), but my Ryzen's are doing a great job PS:- I've just started a couple of 4K tests, and the pre filter's REALLY slow the process down, but "straight" SMDegrain's seem to do well. Have I ever said how much I HATE 4K encoding, even with several Ryzen 9's grinding away
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) Last edited by Pauly Dunne; 17th May 2022 at 04:37. |
|
17th May 2022, 17:59 | #19469 | Link |
Registered User
Join Date: Jan 2018
Posts: 16
|
Ripbot has worked perfectly for me for years. Just today I tried starting it up and immediately see a dialog:
Runtime error 217 at 00411E42 There is an entry in Event Viewer showing the following: Faulting application name: RipBot264.exe, version: 1.26.1.0, time stamp: 0x2a425e19 Faulting module name: KERNELBASE.dll, version: 10.0.19041.1706, time stamp: 0x40a40d01 Exception code: 0x0eedfade Fault offset: 0x0012c3a2 Faulting process id: 0x3de8 Faulting application start time: 0x01d86a0e3aa8b2ba Faulting application path: C:\Program Files (x86)\RipBot264\RipBot264.exe Faulting module path: C:\Windows\System32\KERNELBASE.dll Report Id: 559a9f50-9bcf-42cc-89e0-339a3a17fd84 Faulting package full name: Faulting package-relative application ID: What could be causing this? --- Edit: Rebooted and it's working now. No idea what happened, but my PC hung at the "Waiting for System Event Notification service" during shutdown and I physically had to turn it off and back on. Evidently something in the OS got fried. Last edited by Dhry; 17th May 2022 at 18:26. |
18th May 2022, 05:53 | #19470 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 1,003
|
Quote:
I've had a dual Xeon E5-2697v2 (12 cores) setup for a while, and it's worked good 'til now, I "transplanted" it into a case, instead of being directly screwed to a table !! The only thing I changed was the type & amount of RAM. (more) I did a test run with it today, and it was SLOW, an old dual Xeon X5690 (a measly 6 cores) system was twice as fast, which is pretty sad. I've tried a few "thread" settings, but nothing seems to help. I'm running Windows Server 2022, might need to try "normal" Windows. Last edited by TDS; 18th May 2022 at 05:55. |
|
18th May 2022, 12:55 | #19471 | Link | |
Registered User
Join Date: Mar 2011
Posts: 433
|
Quote:
Is the RAM running at a slower speed, that could cause a bit of a difference as well. Once I tuned the threading settings I was getting good performance. But that area where performance was good was pretty narrow. You're probably going to have to do some benchmarking with different thread settings and see where your sweet spot is. Perhaps NUMA is enabled as well. I was testing NUMA on my older Ryzen, I found non NUMA was better for me. |
|
19th May 2022, 08:48 | #19472 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 1,003
|
Quote:
I'm still not convinced that SMDegrain needs to added to the Threads list !!! The RAM is the same speed, 1066, it was what came with the Dell R720's, I didn't need 128Gb in them, so I borrowed 64Gb. I'm doubtful that it's a Windows Server 2022 issue.... Correct me if I'm wrong, but I don't think NUMA is an option with Ryzen CPU's, it's called something else, but I don't think it works the same way, but I'm running Intel Xeon's, and have the NUMA function disabled. But one very annoying error is randomly happening AGAIN, x265 [error]: unable to open input file, and with x264, as well I did my 1st encode on one of the R720's, and initially got this error, but then I realised I hadn't updated vc-redist(s), after that it worked, but was a bit slow & stalled a few times, but it was runnung @ 100% CPU usage, 'til it stalled, but it completed the chunk, and then with the next encode the error returned, so I turned it off in disgust !!!! |
|
20th May 2022, 13:13 | #19473 | Link | |
Registered User
Join Date: Mar 2011
Posts: 433
|
Quote:
I think NUMA is NUMA. I just built a 24 core EPYC server and Linux reports NUMA is available. I noticed that I got the unable to open input file on certain scripts in Pauly Dunne's tools. The SMDegrain CPU is working fine for me and that is what I'm doing testing with. I'm seeing 7-12 FPS with 4K with SMDegrain depending on options. |
|
20th May 2022, 21:20 | #19474 | Link |
Registered User
Join Date: Mar 2011
Posts: 433
|
SMDegrain Needs to be Added to RipBot264
So I've spent a few days getting to know SMDegrain and it's options compared to MDegrain. At first using Pauly Dunne's custom scripts for SMDegrain did not provide better results than MDegrain and were in fact worse size wise than just using MDegrain. Pauly encodes for bit rate and I prefer Constant Quality since the file size actually tells me how clean the output is.
I found good documentation for SMDegrain here: http://avisynth.nl/index.php/SMDegrain Originally I was testing with a 15 minute clip from Invasion of the Body Snatchers in 4K. But the better test case was to use Blade Runner in 4K since the 15 minutes clip from the beginning has lots of grain, haze, cigarette smoke, etc, plus it will show off color banding if there are problems. My test results: Source: Blade Runner 1982 4K - 15 Minutes cut from 2:58-17:58 Source Size: 4.0 GB Constant Quality 18 6.1 GiB: No Degraining: (Note, file size is larger than the source because of so much grain) 2.2 GiB: MDegrain2 thSAD=400 1.6 GiB: MDegrain3 thSAD=600 Note: I Find MDegrain3 can cause problems with example of a close up a face, you can see the facial texture, but if they suddenly turn their face, it blurs, then you can see the texture again when the turn finishes. Very annoying to watch and as such, I only used MDegrain3 on the absolute grainiest sources. 4.1 GiB: SMDegrain(video) (no options) (See docs for the default options) 2.8 GiB: SMDegrain(video,tr=2,thSAD=400,thSCD2=80) 2.8 GiB: SMDegrain(video,tr=2,thSAD=400,thSADC=200,thSCD2=80) 1.7 GiB: SMDegrain(video,tr=2,thSAD=800,thSADC=400,thSCD2=80) 1.9 GiB: SMDegrain(video,tr=2,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true) 1.0 GiB: SMDegrain(video,tr=4,thSAD=800,thSADC=400,thSCD2=80) 1.1 GiB: SMDegrain(video,tr=4,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true) 828 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80) 3.1 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=-1,refinemotion=true) 1.2 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=0,refinemotion=true) 967 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=1,refinemotion=true) 767 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true) So increasing the "tr" value is essentially which MDegrain you want to use. In the case above, from 2-8. The file size was actually quite good using a tr of 8. The contrasharp provides sharpening. I found adding the contrasharp provided a nice detail enhancement but it really needed to be paired with using a prefilter which blurs out the grain a bit before sharpening. A prefilter of -1 provides no filtering, 0 is light, 1 is medium, and 2 is strong. Blocky sources need to use 3 for the prefilter. Refinemotion helps with detecting motion, but if the prefilter is too strong it can cause problems. I watched the output of the 767 MiB file this morning and compared it to the source and the difference is outstanding. Atak, please add SMDegrain to RipBot264. It seems to be a swiss army knife of sorts. You will probably need to give perhaps three presets and hopefully and Advanced that lets you tune some of the parameters such as tr, thSAD, contrasharp, prefilter, refinemotion, etc. I almost think SMDegain could be so good you can retire MDegrain at this point. You can download the 15 minute Blade Runner source file and the 767 MiB file to see the differences from here: https://cloud.chrisdos.com/s/mdMDjgQMbMCdaes The link expires in two weeks. I've ripped some of my grainiest 4K movies to try things out on and watch them and see how it fairs. Pauly Dunne, thank you for the hard work of providing the SMDegrain scripts and tools. I'll try and take SMDegrain pieces and integrate them into to RB's default tools folder so I can turn on auto updates again. |
21st May 2022, 03:03 | #19475 | Link | |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
Quote:
WOW, you HAVE been busy, well done !!! I am SO SO happy that you have pursued this, I knew that you would NOT be disappointed with SMDegrain, once you got your head around it. So I have compared & cross referenced some of your "scripts" with "my" filters names, and it seems that you are mainly using the E-H (extra hard) settings, and just changing the "tr" values. There are so many others to choose from with lower settings, already. For example :- 3.1 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=-1,refinemotion=true) is basically the SMDegrain E-H Minblur#-1 filter, and so on... 1.2 GiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=0,refinemotion=true) SMDegrain E-H Minblur#0 967 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=1,refinemotion=true) SMDegrain E-H Minblur#1 767 MiB: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true) SMDegrain E-H Minblur#2 So if your starting from scratch with your own SMDegrain scripts, they are already there !!!, you just need to look in the Custom folder. Should I rename them to something a little more explanatory ?? Now I know you're no stranger to modifying scripts, so you can go crazy modding any of these to suit your needs, and just save them in the Custom folder, and delete any that you don't want. So I will take this opportunity to provide some basic "hardware" requirements for some of the SMDegrain variants :- Any BM3D CPU filters requires a CPU with AVX2 instructions, so that means basically most hi-end Ryzens, Threadrippers, etc, but very few Intel's, maybe Gen 12. I would suggest you check the specs of the CPU's you use !! Any BM3D CUDA filters requires an nVidia GPU (faster the better). Any DGDecNV filters requires an nVidia GPU for the nvenc/nvdec, functions. Any KNLMeansCL filters requires OpenCL, either nVidia or AMD GPU's. Most of the other filters are good to go. Also, if you want to suss out more "Dogway" modded SMDegrain stuff, go here :- https://forum.doom9.org/showthread.php?t=182881 The link you posted for SMDegrain info is now a little dated, due to Dogway's mod's. (that's what I use) And don't ignore some of the other filters in the Custom list... I would be shocked if Atak did do anything to implement some SMDegrain filters, but never, say never. And as for Auto Updates, just disable it, to avoid problems !!!! Cheers & thanks PS:- I have downloaded your Bladerunner sample, will take a close look.
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) |
|
21st May 2022, 03:19 | #19476 | Link | ||
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 1,003
|
Quote:
I did some more testing on my "slow" dual E5-2697v2 yesterday, and tried all sorts of setting's to see if I could get some performance back. And I ended up changing a couple of x265 commands, and filter settings. I even tried NUMA & no NUMA, and with NUMA enabled, made a very noticeable speed increase, and if you only use 2 encoding port address's, you have one for /node 0, and one for /node 1...nice. Speaking on NUMA, Ryzens don't have it, BUT Threadripper's & EPYC's, apparently do.... And I still don't think SMDegrain needs to be added to the filter thread list, I actually noticed a slight increase in speed, with it not even IN the list, in the RipBot264.ini file. And I think I had a bit of a brain fart with the R720 the other day when I got that x265 error....as it had previously worked ok on an encode, but the next encode, I got the error...turns out I was using a different filter which required AVX2, which the Xeon's don't have..... Quote:
|
||
21st May 2022, 03:28 | #19477 | Link | |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
Quote:
You just need to convert the audio to E-AC3 @ 1536kbps. NICE, cheers.
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) |
|
21st May 2022, 12:14 | #19479 | Link |
Registered User
Join Date: Mar 2011
Posts: 433
|
My main RP machine is a virtual machine with 24 cores and 48 threads. The Linux KVM Hypervisor has dual E5-2680 v4 @ 2.40GHz (14 cores) and passes 24 cores and 48 threads to the VM. I have a first gen Ryzen 1800X that I use for an encoding server. With those I see 7-12 FPS with 4K just using SMDegrain CPU and no GPU assistance.
|
21st May 2022, 18:45 | #19480 | Link |
Registered User
Join Date: Mar 2011
Posts: 433
|
SMDegrain - Issue and Solution
So I encoded Invasion of the Body Snatchers 4K over last night and I started watching it this morning and I encountered a problem in one of the scenes in the movie.
It's a dark scene with two people talking and their faces are in the light. If you watch the women's cheek you will see that seems degrain for a quarter second, then not degrain, and back and forth. It was very odd. I extracted that 90 second scene out and looked it it in detail and the source has some odd grain around the women's cheek and that is probably where the problem occurs. I ran a bunch of tests using different options against the sample in order to find out how to solve the problem. Reducing the block size from the default of 16 to 8 solved the problem. As a bonus the file size was 11% smaller than before using the default block size. Encoding takes a hit since I'm essentially quadrupling the number of blocks to analyze. Encoding was about 30-40% slower. Link to the sample source and the two encoded files (link valid for two weeks): https://cloud.chrisdos.com/s/6JkTfF2etSo8Ln8 I've changed my scripts to use a block size of 8 now: SMDegrain(video,tr=8,thSAD=800,thSADC=400,thSCD2=80,contrasharp=true,prefilter=2,refinemotion=true,blksize=8) Pauly Dunne: You have a lot of scripts. Most would not run on my system since I don't have a GPU. So you may want to add a GPU/CPU in the name or provide a different folders with GPU and CPU optimized scripts. You may also want to condense the scripts into an advanced vs simplified. Perhaps just have 4-5 scripts from very light to very hard and then all the other scripts in a different folder for those that want to try all the different options and alternatives to SMDegrain. |
Tags |
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360 |
|
|