RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY

[Atak_Snajpera]

About "shiftin content": Are you sure this is the same frame? Maybe its a little movement in the picture, since only lower parts of the picture seem to shift, the boots stay where they are... Maybe hgdn3d takes previous and following frames into account and therefore shifts frames in the time-domain - so the shown fram actually is the frame number +1... ?

KNLMeansCL: Whats Default Setting? In Ripbot, default here is "0" which seems to do nothing at all... "2" and above on the other hand shows clear load on the GPU and very visible noise removal in preview.

I'm 100% sure that frame is the same. Here another example
https://i.postimg.cc/7hy2Y9Tq/hqdn8.png

Default settings for KNLMeansCL are d=1, a=2, s=4, h=1.2

The main problem with KNLMeansCL is that it applies blurring blindly to all areas! Take a look at the jacket.

Screenshots in 1080p (postimage automatically resizes images to 720p)
https://www.mediafire.com/file/u32eza1lwro1eb4/denoisers.7z/file

[ReinerSchweinlin]

Object Shifting: I was referring to the cartoon screenshots, not the live-movie scene :)

Default settings in Ripbot for Setting "2" here is: video,d=1, a=2, s=4, h=2 ... ???

live-szene from movie: Blurring: Yes, You are right, it takes away some details on the darker part of the jacket... On the other hand: medegrain changes the contrast ratio of some shadows in the picture. Look at the dark parts of the wood in the background - the contrast is significantly lower in these parts - if you change very quickly between orig/degrain you see it... Do the same with nlmeans/orig - the picture overal is more "original"

[Atak_Snajpera]

Object Shifting: I was referring to the cartoon screenshots, not the live-movie scene :)

Default settings in Ripbot for Setting "2" here is: video,d=1, a=2, s=4, h=2 ... ???

live-szene from movie: Blurring: Yes, You are right, it takes away some details on the darker part of the jacket... On the other hand: medegrain changes the contrast ratio of some shadows in the picture. Look at the dark parts of the wood in the background - the contrast is significantly lower in these parts - if you change very quickly between orig/degrain you see it... Do the same with nlmeans/orig - the picture overal is more "original"

Detail retention is MUCH more important than some slight changes in contrast. MDegrain2 is overall much more "intelligent" denoiser than KNLMeansCL. I really do not like when denoising is applied to whole frame regardless of the motion.

PS. Shifting in HQDn3d always occurs regardless of the content. Check yourself if you do not believe me.

[ReinerSchweinlin]

PS. Shifting in HQDn3d always occurs regardless of the content. Check yourself if you do not believe me.
From first glance, it might have been that these are two different frames, since the moving of the object is not everywhere in the frames. There are NR Filters out there which work on the temporal domain and therefore rely on previous frames and introduce a time-offset of some frames. So it could have been possible that after segmented encoding HQDN3D misses a frame or adds one, so the video gets "shifted" forth or back about one frame..

This was my assumption - I´ve seen this a lot - thats why I asked about the frames.

Quick tests with HQDN3D showed no object moving here - but that doesn´t prove anything, maybe the source just didn`t lead to this behaviour, others would...

It has nothing to do with "not beleiving" :) The shift is obvious in your screenshots.

[byteshare]

he always come thru with flying colours :)
He really does...I've dealt with several devs on other forums and lets just say their first reaction isn't how can I help you but how are you doing things wrong...and 5 days later I find a nice update without comment from them.
So yes, RipBot is awesome.

[byteshare]

I think it is time to get rid of hqdn3d denoise filter. I really do not like how this filter shifts content of the frame. KNLMeansCL does better job here. Open images in separate tabs and you will see what I mean.
I agree with these conclusions :)
If you're doing older cartoons, have you considered a line sharpening filter and/or a line darking filter?
I use a few depending on the situation.

[slalom]

Have you tried TimeCodes?
https://forum.doom9.org/showthread.php?p=1849546&highlight=timecodes#post1849546
Why should I?
Is it rellevant to this error?
FFAudioSource: No audio track found
(D:\Temp\Ripbot264temp\Job1023\getinfo.avs, line 4)

[mdchaser]

You said 44 cores so I assumed that you have dual E5-2699V4.

My bad, I used the wrong nomenclature. That would be 44 threads, not cores. Each system is running a single 22 core cpu (2699V4) with a single numa node for the entire system. I just built a third system today and have the same results, double click the encodingserver.exe file and windows just looks at you funny :). I have been able to do some encoding using an old file from late 2017, it seems to work fine with the current ripbot version.

Thanks!

Jeff R.

[mdchaser]

And now?
http://www.mediafire.com/file/1btksnqw32of8b9/EncodingServer.exe/file

Brilliant, that did it! Encodingserver.exe now starts up on every system I try it on, thank you! I will definitely be donating again this Christmas.

Thanks!

[guest]

Hey guy's,

I would be interested to hear your comments on which media player software you use & recommended for 4K playback on, of course, a 4K monitor.

I have been using VLC for years, but I have come to realise that it doesn't seem to be "up for the job", and even tho there are fix's out there, they don't seem to fix it (stuttery playback).

I also tried PowerDVD, and it seems ok on some files, but not all :(

So I did a Google, and that revealed a few good players :-

MPC-HC (that Atak uses within Ripbot) (seems to play 4K very well on a non 4K monitor)

MPV (haven't really tested this one, yet)

Potplayer (also seems to play 4K very well on a non 4K monitor)

So I've probably answered my own question, but would be interested what other's use :)

Cheers

[byteshare]

Why should I?
Is it rellevant to this error?
FFAudioSource: No audio track found
(D:\Temp\Ripbot264temp\Job1023\getinfo.avs, line 4)

I thought you were talking about this part of the response:
Originally Posted by Ryushin
So I'm still encountering this issue.

I rip a 4K disc to the hard drive that has seamless branching, such as Incredibles 2. I then use Ripbot to pull in the movie and process it. The output will have out of sync audio towards the end of the movie. I think this only happens on 4K discs that are towards 2 hours or longer.

Time codes should fix your audio being out of sync. You can always set RipBot to "Skip demuxing process" and manually mux the subs later. Personally what I do but with a batch file. I don't handle audio or subs in RipBot but mux them back from source with a batch file. One of the main reasons for this is so that I can keep the MKV Title info.

Hey guy's,

I would be interested to hear your comments on which media player software you use & recommended for 4K playback on, of course, a 4K monitor.

I have been using VLC for years, but I have come to realise that it doesn't seem to be "up for the job", and even tho there are fix's out there, they don't seem to fix it (stuttery playback).

I also tried PowerDVD, and it seems ok on some files, but not all :(

So I did a Google, and that revealed a few good players :-

MPC-HC (that Atak uses within Ripbot) (seems to play 4K very well on a non 4K monitor)

MPV (haven't really tested this one, yet)

Potplayer (also seems to play 4K very well on a non 4K monitor)

So I've probably answered my own question, but would be interested what other's use :)

Cheers

I personally use VLC, MPC-HC, and Kodi for 99% of my video playback and/or testing. Granted I don't do 4K all of the time but I've used all 3 of those when I do.

[byteshare]

add /restart-if-no-progress to EncodingServer.exe commandline.
This command will restart encoding on server if ffmpeg.exe and x264.exe/x265.exe are not using cpu cycles for 1 minute.

Basically I'm checking USER TIME for specific process. This value increases if application is working.
https://i.postimg.cc/SKf2n6VJ/Untitled-1.png

Had the issue again:
https://imgur.com/a/K0Pn5pi
(cut off columns in the photo. I put this int he description for the photo as well, but: Columns are: Process, CPU, Private Bytes, Working Set, PID, Description, Company Name, Session, CPU Time, CPU History, and Cycles)
I'm not sure where you got that User Time in your picture, but I had to use Process Explorer, because in Windows 10 the Task Manager is trash. Incase you wanted User Time Vs. Total time there were also most the same:
ffmpeg.exe User Time: 0:02:23.687 / Total Time: 0:02:24.703
x265_x64.exe: User Time: 0:01:15.984 / Total Time: 0:01:17.031
Full details in the image above (2nd Image)

Looks like FFMPEG gave up at some point?
The other segments were doing ~13-14FPS, or about 107s per segment
There are 23 segments and it got stuck on 22, but 23 finished.

My command line:
/port 1000 /minimize /priority low /restart-if-no-progress 8
/port 2000 /minimize /priority low /restart-if-no-progress 8
Only using 2 servers

Using Process Explorer I killed the Process Tree for the "EncodingServer.exe" that was stuck and the next server picked up and finished the job, but the whole thing was stuck for many hours.

If you needed more info than this let me know and I can gather next time it happens.

[Atak_Snajpera]

I suspect that EncodingServer.exe died as well. That would explain why EncodingServer.exe has not restarted child processes. Please confirm my theory!
Basically kill ffmpeg.exe and x265_x64.exe and see If EncodingServer.exe will start encoding again.

[byteshare]

I suspect that EncodingServer.exe died as well. That would explain why EncodingServer.exe has not restarted child processes. Please confirm my theory!
Basically kill ffmpeg.exe and x265_x64.exe and see If EncodingServer.exe will start encoding again.
Sounds good, I'll wait for it to happen again and only kill those two.
Thank you for your time.

[Atak_Snajpera]

Sounds good, I'll wait for it to happen again and only kill those two.
Thank you for your time.

I'm 99,9% sure that you will see EncodingServer (not responding) message.

[guest]

I suspect that EncodingServer.exe died as well. That would explain why EncodingServer.exe has not restarted child processes. Please confirm my theory!
Basically kill ffmpeg.exe and x265_x64.exe and see If EncodingServer.exe will start encoding again.

I will do the same....pretty sure I don't get any messages :(

[byteshare]

I will do the same....pretty sure I don't get any messages :(
Same, I don't see an error when this happens, but that makes sense if the EncodingServer.exe is stuck, which reports the errors.

[Atak_Snajpera]

This is what I meant
https://docs.microsoft.com/en-us/windows/desktop/win7appqual/images/preventinghangs-ghostwindow.gif

[guest]

This is what I meant
https://docs.microsoft.com/en-us/windows/desktop/win7appqual/images/preventinghangs-ghostwindow.gif

Is that suppose to be an example of the error message ???

And that also looks like W7...I think you'll find that the majority of "us" user's are running W10 or Server '16 / '19, and if I recall those OS's throw up messages down in the bottom right hand corner of the screen, or maybe off the Taskbar.

I am now only running one W7 machine, as I haven't noticed any advantages or disadvantages using W7 over W10 with Ripbot.

You might have to do some test runs on a VM W10..

I'm sure I did some test's for you a couple of weeks back, by suspending most of the components used by the Ripbot process, and those test's revealed that if you suspend either ffmpeg, x264 & x265, that it will indeed restart after the prescribed time in the Encoders commandline instructions...I have mine set @ 2 minutes, byteshare has it @ 8 (for example).

Maybe it's a case of monitoring Encoding Server the same way you did it for ffmpeg, etc, and that might make it almost "bullet proof" reliable :)

[guest]

My command line:
/port 1000 /minimize /priority low /restart-if-no-progress 8
/port 2000 /minimize /priority low /restart-if-no-progress 8
Only using 2 servers

Hey, I'm curious why you haven't got a numa "command" in there somewhere ???

What CPU(s) are you running ??? you could at least be using NUMA 0 !!

[byteshare]

Hey, I'm curious why you haven't got a numa "command" in there somewhere ???

What CPU(s) are you running ??? you could at least be using NUMA 0 !!
Using an AMD R7 1700, so I just let it default to Numa 0. It defaults to Numa 0 if you don't define it.

[guest]

I suspect that EncodingServer.exe died as well. That would explain why EncodingServer.exe has not restarted child processes. Please confirm my theory!
Basically kill ffmpeg.exe and x265_x64.exe and see If EncodingServer.exe will start encoding again.

Just to bump this post for an update...

So I used Ripbot for the 1st time in over a week, today, using the newest EncodingServer 1.12.6.0, and the very first job stalled on Port 2000 !!!

But before I say anymore, I still can't understand why it won't start encoding after adding Job's, until you restart Ripbot :mad:, this has only started, recently.

OK, so back to the stall, I went into Process Hacker, and suspended ffmpeg, but it didn't seem to have any affect.

There were no pop up warnings, either, so that might also indicate that it's not ffmpeg, x264 or x265...

Back to the Encoder Client window, those ON/OFF buttons for each server control the Encoding Server, correct ??

So, if those button's don't do anything wouldn't that mean that Encoding Server has a major problem ??!!! You can keep pressing the OFF button forever, and it does nothing !!!!

Aborting the whole Job seems to be the quickest & best "fix".

I can't explain it any other way, I'm not an "IT" person !!

OK, a couple of hours later, I have had it happen several more times, but on the Client's, not the main PC (Server, I guess it is)...I cannot diagnose it with Process Hacker, everything appears to me, is still working, but one Encoding Server has definitely "locked up", and there's really only one course of action...re start Ripbot, and hope it doesn't happen again, anytime soon.

And as a result of aborting & restarting, the job starts from the beginning :(, and it show any other jobs that have been completed, as not being done.....:confused:

[Atak_Snajpera]

Maybe it's a case of monitoring Encoding Server the same way you did it for ffmpeg, etc, and that might make it almost "bullet proof" reliable
https://i.postimg.cc/HLpVb9zt/Untitled-1.png

http://www.mediafire.com/file/aw8gqz7snt7twlb/EncodingServer.exe/file

[guest]

https://i.postimg.cc/HLpVb9zt/Untitled-1.png

http://www.mediafire.com/file/aw8gqz7snt7twlb/EncodingServer.exe/file

OK, well I can't read your mind, but what I did was download and overwrite the newer EncodingServer.exe.

I only did one encode today, and after a tiny little hiccup right at the start (probably my fault), it run faultlessly....the ONLY different thing that happened (and I wasn't watching when it did), but I returned to notice a "different" looking icon in the Taskbar, so I clicked on it, and there were 4 instances of that "Supervise Process" screen in your previous post, one for each of the servers/ports I had going.

Does that mean that there was a stall on all 4, and they were all started automatically ???

[Atak_Snajpera]

Does that mean that there was a stall on all 4, and they were all started automatically ???
SuperviseProcess.exe is always started if you use /restart-if-no-progress switch. If application does not respond for 1 minuted then it will be automatically killed.
Process Tree
https://i.postimg.cc/7ZDZ6K41/Untitled-1.png

Example using different application for testing purposes
https://i.postimg.cc/kgyvHYwB/Untitled-1.png

[guest]

SuperviseProcess.exe is always started if you use /restart-if-no-progress switch. If application does not respond for 1 minuted then it will be automatically killed.
Process Tree
https://i.postimg.cc/7ZDZ6K41/Untitled-1.png

Example using different application for testing purposes
https://i.postimg.cc/kgyvHYwB/Untitled-1.png

OK, I'm a little confused (which doesn't take much with this sort of stuff), but I thought the restart-if-no-progress was the "monitoring/control" of ffmpg, x264 & x265 if they stalled (which seems to work very well)...does this monitor Encoding Server, and if it "stall's", this kill's it, then what start's it again ?? or is that what this does ??

And this must be what you've incorporated in the Encoding Server you provided, yesterday.

[Atak_Snajpera]

It does both now. Monitors ffmpeg.exe/x264/265 and also MAIN application. SuperviseProcess.exe monitors EncodingServer.exe and EncodingServer.exe monitors ffmpeg/x264/x265

[ReinerSchweinlin]

I´d love to test the new version of encodingserver.exe, but sadly, my virusscanner thinks it is faulty, so I have to wait till I am at another system with no scanner or where I can turn it off :)

[Atak_Snajpera]

I´d love to test the new version of encodingserver.exe, but sadly, my virusscanner thinks it is faulty, so I have to wait till I am at another system with no scanner or where I can turn it off :)

Problem?
https://www.virustotal.com/#/file/92aa5b5a882d7ecae7e6a8b630dae817ee7604f8da1386ab18ecff40003d8eb6/detection

[ReinerSchweinlin]

Already did a check on virustotal, seems fine (two heuristic notices, not worth it..), but I can´t switch virusscan of here (workstation..)

On another note concerning the OPENCL on XEON Issue: I found "GPU" in the avs script in the KNLEANSCL Line - changed it to "auto", works fine now. Not sure how this affects all other machines with real GPUs.

[Atak_Snajpera]

you can also manually specify device type now.
/knlmeanscl-opencl-device-type CPU

[ReinerSchweinlin]

Yes, but wouldn't that force all of the clients to the CPU? Or could this embedded for each server manualy?

[Atak_Snajpera]

EncodingServer.exe /knlmeanscl-opencl-device-type CPU

[ReinerSchweinlin]

How can I set up 4 serves on one machine like this? 2 on UMA Node 0, 2 serves on Node 1 ?

[Atak_Snajpera]

go to settings -> Distributed Encoding -> Click spanner icon -> Command line help

[ReinerSchweinlin]

wonderful :) Thanx!

[ReinerSchweinlin]

Thanx for the hint.

[Ryushin]

Atak, cannot begin to tell you how much I enjoy the feature of moving jobs around now. Makes my life a lot easier.

A couple of ideas for features.

When selecting the item for a new job, allow a setting to change the display job cutoff time from the default of 15 minutes to another value. I'm having to use Handbrake to pull in a job to encode the specials. If I can drop the time down to 5 minutes or so so I can pull in those specials into RibBot.

Since I'm doing a lot of TV shows. It would be so nice to highlight a lot of jobs to bulk change an option. Such as the constant quality number, degraining, deinterlace, etc. It can take quite awhile to edit 50 jobs or more to change one setting.

BTW, your interface works fine with a job number over 1000. :)

[Atak_Snajpera]

When selecting the item for a new job, allow a setting to change the display job cutoff time from the default of 15 minutes to another value.
I really do not understand this.

[byteshare]

allow a setting to change the display job cutoff time from the default of 15 minutes to another value. I'm having to use Handbrake to pull in a job to encode the specials. If I can drop the time down to 5 minutes or so so I can pull in those specials into RibBot.
Are you saying that videos shorter than 15 min aren't being handled by RipBot? I do jobs are are less than a minute sometimes.
Would you post some screenshots of what you're talking about? (use an image hoster like IMGUR)

[Wishbringer]

Maybe he means displaying playlist from Blu-ray while importing.

[Ryushin]

Maybe he means displaying playlist from Blu-ray while importing.

Yes, this is what I meant. When importing a playlist, anything under 15 minutes is not shown.

[gryph]

First, thank you so much for this program!
I've been using RipBot for years now.
However, for a long time now, there's been some sort of bug - havn't been able to find any reports of this, so here I go...
I use Ripbot to hardcode subtitles and set a fixed video bitrate.
However I have to edit every single subtitle file (.srt/.ass) with an extra line in the very end to assure I get the "original" last line of sub hardcoded.
Which ever program is handling the subtitles, is deleting the very last line in the process.
So adding a custom new last line have been my fix.
Not sure exactly when this started to happen - but it's been going on for a long long time (1 year+ at least I'd guess).
Prior this issue didn't occure and there was no need to add an extra line of subtitle.
I ofc. always keep Ripbot up-to-date. So question is, is there a bug somewhere in the program - or am I missing something?
(btw - I run on Windows 7 machines only)

Fingers crossed for a fix... would save me from having to edit every single subtitle file before encode.

Cheers

[ReinerSchweinlin]

you can also manually specify device type now.
/knlmeanscl-opencl-device-type CPU
Just tried it out - works very well, Thank you!
Of course, it´s slow... But on some old TV-Shows with low resolution (old MPEG2 Captures) it works quite well with the DUAL Xeon Machines - and this way I don´t have to worry to manually tweaks settings in the swarm, I simply hit "encode" :)

[ReinerSchweinlin]

Now I know this is probably a really basic noob question,

Can someone explain why when you re-encode an x264 to x264 with just some basic settings, it doesn't take a very long time.

However, when you re-encode an x265 4K to x265 4K, it takes such a L-O-N-G time.

Why when it's already x265 4K !!!!

Downsizing from x265 4K to x265 1080p doesn't take long.

There has to be a faster way to do this, especially as it's becoming a lot more common place.
The codec used for the source is not the point - the time it takes to decode is muuuuuch smaller then to encode.
So if the source already is h265 makes no difference.
What takes time is the new encoding - the more advanced the codec (x265>X264), the more pixels (4k>1080p) and the higher the encoding settings is what takes up time..

Encoding 4K in x265 really needs a lot of horespower, no matter what the source once was...

[Ryushin]

However, when you re-encode an x265 4K to x265 4K, it takes such a L-O-N-G time.

Why when it's already x265 4K !!!!

Downsizing from x265 4K to x265 1080p doesn't take long.


4K is 4 times the size of 1080p. So it should take 4 times longer to encode the same video, regardless if you are encoding x264 or x265. Encoding with x265 is just much more processor intensive. By a lot. Older processors which did well with x264 have a real hard time with x265.

My older Opteron server with a true 48 cores and 256GB of RAM did a great job with x264. But with x265 it has crossed the threshold of too much power consumption vs its output rate for x265. So I'm getting rid of it soon.

AMDs new Zen based chips are great with x265. I don't know if anything is using a GPU for x265 encoding yet.

Nature of the what we do. When blu-ray came out, encoding was very slow compared to DVD. Now we are in the same evolutionary step with 4K and x265.

[ReinerSchweinlin]

.. I don't know if anything is using a GPU for x265 encoding yet.
Yes. Intel, Nvidia and AMD have hardware-encoding which is getting better. For realtime game.streaming it is the way to go. For offline Video-content, it slowly catches up to Software-Encoders.. Quite a lot of software packages enable these encoders already.
I have been using Intel Kaby Lake architecture for hardware-encoding into h265/10Bit - which works fine for many usecases. A 60 Euro CPU is able to handle one 1080p Stream way faster than realtime..

As we are talking - any chance ripbot will support Intel Quicksync?

[Ryushin]

Unfortunately upgrading to hi end cpu's & gpu's is really not an option, these days :(

Actually, with last years Zen 8 core CPU, I'm seeing 5-7fps with 4K x265 encodes. Seeing as how you can get the CPU for around $250 US, that at least brings it within reason. Add MDegrain though, and I'll get about 1.5 to 2fps. So about a day for each 4K encode.

[Atak_Snajpera]

Hi Atak, I thought I'd bump this one up a page or 2, as I have a problem with this that you may want to know about.

Today I had a lot of PC's going to see how it would perform doing a large 4K x265 encode (as per previous post), and I noticed on one of my "server box's", that an Encoder Server stalled (using the latest version 1.12.6.0), so I opened up the Supervise Process window for the stalled port, and there was no activity, it wasn't progressing every second or so, like the others do, monitoring the process's. So it had stalled, as well.

I left it like that for several minutes to see if it would start again, but it didn't so I killed it, and turned that pc off.

Having said that, I didn't any issues with any of the other machines, except one PC just stopped working, but that's a different issue, altogether.

So there's still some weird issue, in there somewhere.

One of your PC is simply unstable. Period. This can not be a coincidence that two different applications (simple console program and more complex application with GUI) just hang in the same way simultaneously. Do not expect from me to fix programmatically your hardware issues.

[Atak_Snajpera]

v1.23.2 Changelog
Added: KNLMeansCL [Adaptive] denoiser. (denoising strength is being reduced according to luminosity of the pixel)
Added: /avisynth-prefetch-threads and /knlmeanscl-opencl-device-type to supported switches by EncodingServer.exe.
Added: /restart-if-no-progress switch also activates extra process (SuperviseProcess.exe) in order to detect any hangs directly in EncodingServer.exe.
(logs are stored in %TEMP%\SuperviseProcessLog.txt and %TEMP%\EncodingServerLog.txt)
Added: Auto reconnect option in EncodingClient.exe
Changed: /knlmeanscl-opencl-device to /knlmeanscl-opencl-device-id and /x264-opencl-device to /x264-opencl-device-id
Fixed: EncodingServer.exe not starting on machines with more than 32 logical processors.

Comparision regarding KNLMeansCL [Adaptive] variant

Source
https://i.imgsafe.org/bf/bf45db39b9.jpeg

MDegrain2
https://i.imgsafe.org/bf/bf46940cf9.jpeg

MDegrain2 + KNLMeansCL(h=4)
https://i.imgsafe.org/bf/bf46ddeee0.jpeg

MDegrain2 + KNLMeansCL(h=4) [Adaptive]
https://i.imgsafe.org/bf/bf4789e165.jpeg

KNLMeansCL(h=4) [Adaptive]
https://i.imgsafe.org/bf/bf47dce349.jpeg

KNLMeansCL(h=4)
https://i.imgsafe.org/bf/bf45f49f3c.jpeg