Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
27th March 2022, 07:29 | #19421 | Link |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
100 downloads :)
Well, it's taken just under a month to reach a substantial 100 downloads of "my" updated build of RipBot, so I hope it's working well for all that have tried it.
I will endeavour to post an update in the next week or so, there has been several significant updates to the components that are used in RipBot. So have at it, and keep watching https://forum.doom9.org/showthread.p...27#post1964827
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) |
27th March 2022, 07:45 | #19422 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 993
|
Quote:
That is one hell of a "farm" you've got going there !!! OK, that IS strange behaviour, for sure, so you've obviously got Distributed Encoding sussed & working, so how many server ports do you enable per pc ???, maybe you need to "load" the 16 cores a bit more to get close to 100%. I'm curious to know if you're using a "genuine" build of RipBot, or are you using the "Pauly Dunne" build ??? There is a possibility that the "older" Tool's in RipBot might be slowing you down....(just thinking out loud). I have several Ryzen's, and they need to be "fed", but again, tbh, I haven't done much 4K encoding of late. Might be a case of trial & error, 'til you get the results you should be getting !!! Cheers |
|
27th March 2022, 19:42 | #19423 | Link | |
Registered User
Join Date: Aug 2020
Location: Pennsylvania
Posts: 82
|
Quote:
Since I posted that original message, I have found a "fix" so to speak. When I start up the 16 core machines in the startup script I added /affinity FFFCFFFC to the command line. This stops Ripbot from using one of the cores on CCD0 and CCD1. I did that to balance out the thermals on the die (seemed logical). Once I did that my fps jumped back up to where I would expect them to be on the 16 core(now 14 cores used). Where before the 16 core would show 2-3 fps compared to the 3900x around 4 fps, the 5950x's are now in the 5-7 fps. Disabling the 2 cores didn't seem to make much of a difference either in the lower resolution stuff so I just leave them running 14 core for Ripbot all the time. Still not sure what is being flooded that kills the performance only on full frame 4k, but for now I am happy. Only thing that is a little strange is since I did this I see a stalled encoding server a little bit more often, but the server resets and continues on without messing up the entire encode. Full frame 4k in 10bit X265 with some Mdegrain3 for minor cleanup ends up averaging around 40fps and a movie can be done in under an hour, works for me... |
|
28th March 2022, 16:30 | #19424 | Link | |
Registered User
Join Date: Aug 2020
Location: Pennsylvania
Posts: 82
|
Quote:
the 16 cores are all running on a mixture of gigabyte and asus x570 motherboards with the latest bios installed and also windows 11 with the latest amd drivers. All are running 32 gig ddr3200 memory in XMP mode. The rest are running B450 or B550 chipsets, latest bios and drivers, and the lone intel is running on a Z490. When I saw and researched the /affinity switch for the encoding server command line, my first thought was OK, lets knock them down to 12 core 3900x levels and see what happens. They immediately jumped to 3900x performance or just a tad over. so I added in 2 more cores to their current level of running 14 cores for ripbot. I used this site to give me the hex code to use: https://bitsum.com/tools/cpu-affinity-calculator/ I start each encodong server with this script: encodingserver /minimize /restart-if-no-progress /affinity FFFCFFFC /priority normal As far as settings in ripbot using MA10-default,default profile, Mode=CQ, CRF=18. Audio I usually knock dts down to ac3 or just copy the stream if it is already in ac3 or aac. This way my Plex server can play everything in direct stream with no encoding. In avisynth tab I just do a autodetect crop, then a cpu denoise of mdegrain3 and do somewhere between 100 and 400 depending on the noise level (grain noise just bugs me for some reason). I'll also render any forced subs if they exist. Helps to throw 212 threads at an encode. The under an hour was for a 1 hour 4k TV show I've been doing lately |
|
18th April 2022, 07:33 | #19426 | Link | |
Registered User
Join Date: Dec 2011
Posts: 148
|
Quote:
|
|
19th April 2022, 02:47 | #19427 | Link | |
Registered User
Join Date: Sep 2018
Posts: 11
|
Quote:
|
|
19th April 2022, 05:03 | #19429 | Link | |
Grumpy Old Man.
Join Date: Jul 2019
Location: Out There....
Posts: 692
|
Quote:
Don't forget to disable Auto Updates, it will mess up these builds !!! Good to hear that DE is working for you, I checked some of your previous posts, and most of them where DE issues...hope it stays working for you. And thankyou for the thankyou
__________________
Not poorly done, just doin' it my way !!! Live every day like it's your last, because one day, it will be !! (M$B) |
|
20th April 2022, 20:52 | #19430 | Link | |
Registered User
Join Date: Oct 2001
Posts: 454
|
Quote:
K80 K40 M40 Grid 520 ...... As long as you can set the card to WDDM Mode, you can use it for almost anything (Direct X, Vulkan, etc..)... OPENCL and CUDA are also available in Compute Mode (wich is slightly faster)... Some of the vrey new and big cards only have CUDA (Ampere datacenter ones) and some rare ones are "hard to fiddle" because they only offer certain drivers (some Kepler Grid ones), but for the most part - the stuff that appears on ebay works fine.... |
|
21st April 2022, 02:26 | #19431 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 993
|
Quote:
So with the Dual GPU cards (eg:- K80), do you assign one GPU per port in DE ?? Or can you assign both GPU's to the same port, as to get the maximum assistance from the nearly 5000 Cuda Cores ?? |
|
21st April 2022, 15:24 | #19432 | Link |
Registered User
Join Date: Oct 2001
Posts: 454
|
The K80 is presented to the system as two seperate GPUs. Since Ripbot is only able to adress one, you wont be much faster than with the K20x.... In theory, it would be possible to use both at the same time, but often the overhead of splitting the workloads into two chunks and moving it over the PCI_E Bus destroys the advantage... The lesser data, the more computation, the better the benefit (blender render is almost twice the speed for example)... So the more demanding the filter is, the more benefit from two cards (realsr vulkan implemenatation shows that quite nice.. up to four cards at the same time can be used).
A M40 would speed up things compared to the K20x. If you donīt HAVE to use Grid/Tesla.... GCN Cards are quite strong in OPENCL... A used R9 Fury is around the same price.-tag as a M40 and is a lot stronger in OPENCL computation... (take a look at geebench opencl resukt browser to get a rough estimate) In addition: Unless you have a lot of CPU power, often the Encode-Part is the bottleneck (at least when doins x265 slow or better)... one thing you could try: Set the card to compute mode and turn of ECC, might squeeze out some speed Last edited by ReinerSchweinlin; 21st April 2022 at 15:28. |
22nd April 2022, 01:38 | #19433 | Link | ||||
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 993
|
Quote:
Actually the main reason I wanted to try some Tesla's is that my "best" GPU, an MSI 1080 Gamer X, stopped working, and then I saw some YouTube clips of guys using these Tesla cards for gaming & transcoding, and as the price of new & used GPU's has been so high, you could get certain Tesla cards for cheap, so I opted for a K20Xm. I did ask around about encoding with these, and got the big "thumbs down", saying they were no good for that, but I needed to find out for myself. I was mainly after the CUDA functionality, not the OpenCL, and after a far bit of "testing" RipBot was using it for both. Even though RipBot has no "default" CUDA filters !! So, you mentioned that the K80 was recognised as 2 GPU's, and that RipBot could only address 1...you must be relying on the info under the OpenCL tab, as it does ONLY show 1 device at a time, even if you have 2 GPU's. I have a couple of systems that have an AMD & nVidia GPU, and the only way you can use both with RipBot is the following command(s)..(in the RipBot264.ini file) This one assigns 1 GPU per encoding port... Quote:
Quote:
I also have no idea how to do this :- Quote:
|
||||
22nd April 2022, 11:33 | #19434 | Link | ||||||
Registered User
Join Date: Oct 2001
Posts: 454
|
Quote:
Quote:
But using them as a computation device for filters (in ripbot or whatever..) they are fine... Not the fastest, of course, but for the money, they work good and provide more VRAM than the consumer GTX counterparts.. Quote:
Quote:
Quote:
x265 does not offer this kind of technique - it was investigated, but since the computation is even more complex and the type of math used here is not very suitable for a GPU, its not done in x265 back then by the developers, the idea was dropped... Some newer developments of direct memory access for GPUs and offloading motion estimation (read up Intel AI SDK news on developer forums) might give this idea a new live, but so far, x265 is best on a lot of CPU cores The second thing is using filters with GPU acceleration... This makes sense... IF the Filter is capable of using more than one GPU and the GUI is able to call it, then it will work - but again, most of whats under the hoodd of ripbot is not capable of splitting workloads... If using multiple filters, doing one on GPU1 and the other on GPU2 COULD work, but then you have the Bandwith Bottleneck of movniog stuff over the PCIE BUS... (the K80 Chips canīt "talk" to each other directly, no SLI or similar awareness here... Bottom line - its much easier to get ONE bigger GPU - and much more promising in terms of "getting faster".. Quote:
|
||||||
27th April 2022, 09:24 | #19436 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: Down Under.
Posts: 993
|
Quote:
http://www.digital-digest.com/softwa...4_history.html |
|
28th April 2022, 15:55 | #19437 | Link |
Registered User
Join Date: Apr 2020
Posts: 16
|
Hey, I have Ripbot on a Windows 10 PC and Windows 11.
The Windows 10 PC is where I'd want most of the brute force coming from, I can get encodings going from the Windows 10 PC and also from the windows 11 PC using the windows 10.. However what I cannot get working is adding in the Win11 PC to add to the power when I want to. I double click the EncodingServer exe, however no GUI opens, it goes to Task Manager but nothing to show to be able to actually "run" I've restarted the PC but still no luck, currently RipBot is situated on my D Drive, would I need to have it on C Drive for the EncodingServer to open properly? Or is it a Win 11 issue? Thanks. Edit - Turns out GeForce Experience was stopping it.. Was the only difference between the machines and the server popped up as soon as it was uninstalled. Last edited by VirulentPip; 28th April 2022 at 16:49. |
28th April 2022, 16:09 | #19438 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,812
|
Run Win11 in safe mode and check if EncodingServer opens.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
28th April 2022, 17:51 | #19439 | Link |
Registered User
Join Date: Mar 2011
Posts: 432
|
MDegrain2 and Threading Bottleneck
So I upgraded my server using a used Supermicro motherboard and I ordered two Xeon E5-2680 V4 (14 cores) processors for it from Ebay. Got everything installed on Tuesday.
I configured my virtual machine with to use 24 cores and two threads for a total of 48 CPU threads. Gave the VM 32GB of RAM as well. I am not using NUMA. Run the x265 Benchmark and received a score of 59.2 so I was happy with this. Received Moonfall in 4K from Amazon and I proceeded to add it to RB. Noticed i would have to do a bit of degraining and enabled MDegrain2 with a THSAD value of 200. Started the encoding with two servers and was amazed at how slow the encoding went and how it was not utilizing the CPU. So I did a test encoding with no MDegrain2 and it was fast. So most likely it was the threading problem with MDegrain2. My "Use multiple processing threads" setting was set to the default of 0 which I think means use a thread for each CPU thread which would be 48. So I ran some tests by lowering the number of threads used for MDegrain2 and the results were dramatic: 1 Minute Chunk Size, First Chunk, 1 Encoding Server No MDegrain - 6.38 fps (CPU running about 45-85%) MDegrain2 48 threads (0 for unlimited) - 1.27 fps MDegrain2 24 threads - 1.29 fps MDegrain2 18 threads - 2.31 fps MDegrain2 15 threads - 4.36 fps MDegrain2 14 threads - 4.44 fps MDegrain2 13 threads - 4.38 fps MDegrain2 12 threads - 4.15 fps MDegrain2 10 threads - 3.62 fps 1 Minute Chunk Size, First and Second Chunk, 2 Encoding servers No MDegrain - 4.17 and 12.13 fps = 16.30 fps (CPU running about 92-100%) MDegrain2 15 threads - 3.05 and 3.62 fps = 6.67 fps MDegrain2 14 threads - 3.18 and 4.22 fps = 7.40 fps MDegrain2 13 threads - 3.26 and 3.96 fps = 7.22 fps So I hope this helps someone who has a highly threaded setup and that you will probably have to tune the MDegrain2 thread settings for each machine. |
28th April 2022, 22:14 | #19440 | Link |
Registered User
Join Date: Sep 2009
Location: Ankara
Posts: 23
|
I am waiting for AV1 codec support in RipBot!
https://www.tomshardware.com/news/in...coder-for-cpus @Atak_Snajpera please look to the bottom link: https://gitlab.com/AOMediaCodec/SVT-...eleases/v1.0.0 Last edited by blacksapprow; 29th April 2022 at 09:59. Reason: Link addition for the topic. |
Tags |
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360 |
Thread Tools | Search this Thread |
Display Modes | |
|
|