RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY

[Lycan91]

Hello,

I am using Ripbot264 1.24.1 in DE mode on my 4 computers. However, EncodingServer.exe does not start on my computer with bi-xeon e5-2697v4 (72 logical processors), even with the latest ripbot update (1.24.1). Ripbot works well on my bi-xeon only in stand-alone mode

If I disable one of the two processors (36 logical processors), ripbot works

Do you have an idea to solve my problem?

Thank you

[Atak_Snajpera]

Hello,

I am using Ripbot264 1.24.1 in DE mode on my 4 computers. However, EncodingServer.exe does not start on my computer with bi-xeon e5-2697v4 (72 logical processors), even with the latest ripbot update (1.24.1). Ripbot works well on my bi-xeon only in stand-alone mode

If I disable one of the two processors (36 logical processors), ripbot works

Do you have an idea to solve my problem?

Thank you

Run EncodingServer.exe via command line

EncodingServer.exe /affinity 1

If above command will work then set manually correct affinity mask. There is definitely some issue with cpus with more than 64 logical processors. Unfortunately I have no access to such machines so there is zero chance that I will be able to locate problem.

[Lycan91]

Hello Atak,

Thanks

I started via command line encodingServer.exe / affinity 1. It starts and i try to activate all CPU, however after the CPU32, it is not possible to activate the last

https://image.noelshack.com/fichiers/2019/28/7/1563124173-2019-07-14-19h07-53.png

And we can activate only one of two nodes available

https://image.noelshack.com/fichiers/2019/28/7/1563125891-2019-07-14-19h36-26.png

https://image.noelshack.com/fichiers/2019/28/7/1563125878-2019-07-14-19h34-29.png

In other hand, if i keep only one CPU (I remove one processor), i can activate all logical processors in EncoderServer

If you are interested, i can make some tests... or i can give you a remote access (google or TeamViewer) to see this issue

[Atak_Snajpera]

What does happen if you disable HT in bios?

[byteshare]

x265 3.1+5 is now stable:
http://msystem.waw.pl/x265/
There were some fixes to aq-motion:
https://bitbucket.org/multicoreware/x265/commits/all

[byteshare]

Remember, with QAAC, it only needs some of the binaries from iTunes. On the QAAC site, there's a batch file and when given the iTunes installer, it extracts the needed binaries.

https://sites.google.com/site/qaacpage/cabinet
Stax does that for you I guess.

[Lycan91]

Hello Atak,

I desactivated HT in bios and EncoderServer.exe start with ripbot264 (Without Commandline), it works fine however only 18 logical / physical processors are active (It's normal without HyperThreading, virtual processors are disabled) on each processor, and only one node is active.

https://image.noelshack.com/fichiers/2019/29/1/1563212663-2019-07-15-19h32-39.png

https://image.noelshack.com/fichiers/2019/29/1/1563212669-2019-07-15-19h33-04.png

If we choose the socket 1 in "additionnal server", EncodingServer only keeps 4 processors and doesn't activate node 1, the node 0 is always enabled

https://image.noelshack.com/fichiers/2019/29/1/1563213629-2019-07-15-19h56-17.png

https://image.noelshack.com/fichiers/2019/29/1/1563213712-2019-07-15-20h01-25.png

Do you want i make other tests ?

Bye

[Atak_Snajpera]

1) Download this small program https://www.mediafire.com/file/zwurxc2a1tpey6q/GetProcessAffinityMask64.exe/file
2) Activate HT again in bios
3) Run RipBot264.exe
4) Check PID for RipBot264.exe in Process Hacker/Process Explorer
Example
https://i.imgsafe.org/cc/cc758ae431.png
5) run cmd.exe as administrator
6) GetProcessAffinityMask64.exe HERE_PUT_PID (in my case it was GetProcessAffinityMask64.exe 3932)
7) Show me content of GetProcessAffinityMask64Log.txt file

[stryker412]

Atak, I'm looking at upgrading my current system. I've always purchased Intel in the past but am open to the new AMD chips. Someone suggested the Ryzen 3600X. Are there any performance differences in comparable chips between AMD/Intel in RipBot?

[Atak_Snajpera]

Ryzen 3600x is basically 8700k in video encoding tasks.

Ps. Buy 3600 without X because https://www.youtube.com/watch?v=rDV7aA6arVo

[Lycan91]

Hi Atak,

I activated HT in bios, and i made your tips

When i start ripbot264 (EncodingServer.exe doesn't run in HT mode), i obtained the following log:

:logfile:

ProcessAffinityMask=0
SystemAffinityMask=0
Node=0
ProcessAffinityMaskPerNode=0
SystemAffinityMaskPerNode=0

Then if i run Ripbot264.exe and i forced EncodingServer.exe /affinity 1 (Via cmd.exe), i get the following log

:logfile:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

:thanks:

[stryker412]

What are your thoughts on the 3700x?

https://www.techpowerup.com/review/amd-ryzen-7-3700x/13.html

[Atak_Snajpera]

What are your thoughts on the 3700x?

https://www.techpowerup.com/review/amd-ryzen-7-3700x/13.html

3700x is basically a 9900k in x265. Your money ,your choice.

[Atak_Snajpera]

Hi Atak,

I activated HT in bios, and i made your tips

When i start ripbot264 (EncodingServer.exe doesn't run in HT mode), i obtained the following log:

:logfile:

ProcessAffinityMask=0
SystemAffinityMask=0
Node=0
ProcessAffinityMaskPerNode=0
SystemAffinityMaskPerNode=0

Then if i run Ripbot264.exe and i forced EncodingServer.exe /affinity 1 (Via cmd.exe), i get the following log

:logfile:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

:thanks:

You get zeros in first example because you didn't run cmd.exe as administrator like I said.

BTW. Does EncodingServer start with these commands?
EncodingServer.exe /node 0
EncodingServer.exe /node 1

[Lycan91]

Atak,

In administrator mode, only ripbot264 (Without EncodingServer.exe started due to HT mode)

https://image.noelshack.com/fichiers/2019/29/2/1563309377-2019-07-16-22h34-49.png

https://image.noelshack.com/fichiers/2019/29/2/1563309411-2019-07-16-22h36-38.png

the log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

If i make only EncodingServer.exe /node 0 (via cmd.exe) EncodingServer.exe doesn't run

https://image.noelshack.com/fichiers/2019/29/2/1563310074-2019-07-16-22h47-32.png

However, if i do EncodingServer.exe /affinity 1 - EncodingServer.exe start

https://image.noelshack.com/fichiers/2019/29/2/1563310253-2019-07-16-22h49-57.png

https://image.noelshack.com/fichiers/2019/29/2/1563310363-2019-07-16-22h50-04.png

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=1
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

And now, as EncodingServer.exe run, if i do EncodingServer.exe /node 0 or EncodingServer.exe /node 1, nothing happens either

https://image.noelshack.com/fichiers/2019/29/2/1563311228-2019-07-16-23h06-45.png

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

Nevertheless, i don't know if it's important but i noticed that when i execute EncodingServer.exe /node 0

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=1
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

And if i do EncodingServer.exe /node 1

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

https://image.noelshack.com/fichiers/2019/29/2/1563312130-2019-07-16-23h21-47.png

it's seems that the node number is switched when we do EncodingServer.exe /node 1 or EncodingServer.exe /node 0 twice

https://image.noelshack.com/fichiers/2019/29/2/1563312193-2019-07-16-23h22-54.png

Bye

[duffbeer]

A quick question for you guys. I just upgraded my PC and I have an SSD drive plus a couple of HDDs.
What is the best configuration/settings to make sure that my encoding jobs don't kill the SSD due to many constant files being written?

[Atak_Snajpera]

A quick question for you guys. I just upgraded my PC and I have an SSD drive plus a couple of HDDs.
What is the best configuration/settings to make sure that my encoding jobs don't kill the SSD due to many constant files being written?

I doubt that you will be able to kill NAND in your SSD by just doing encoding stuff.

[SKPN]

Might be easier to do it with MKVToolNix IMO...only works with Keyframes though, so might be better to do with source or your encode depending where the keyframes are.

I use MKVToolNix a lot with stuff that has intro trailers and outro trailers after I encode.

Otherwise you can define the frames to use with AVISynth.

Thanks. That was my fear. Unfortunately, the cuts don't always land on a keyframe, so I'll have to just do the video encoding in RB264 and figure out another solution to cut the audio properly.

[byteshare]

VMAF: https://streaminglearningcenter.com/blogs/installing-and-using-netflix-vmaf-master.html
Would be "neat" to have something like this integrated into RipBot without having to manually do the CSV file steps.

[FuzzyNutz]

When I click the "x" in the top-right corner to close RipBot264, it hangs. I have to force RipBot to close. Deleting RipBot and re-installed it has had no effect. I've been using RipBot for nearly a decade. It starting doing this last year, if i recall correctly.

I'm running Windows 10 Pro, newest version.

[byteshare]

When I click the "x" in the top-right corner to close RipBot264, it hangs. I have to force RipBot to close. Deleting RipBot and re-installed it has had no effect. I've been using RipBot for nearly a decade. It starting doing this last year, if i recall correctly.

I'm running Windows 10 Pro, newest version.

I had some issues with that too. Not sure in your exact case as one of the RipBot updates fixed the issue for me.

[Viper714]

So lately I have been experiencing playback issue on some of my converted movies. Typically it is with older movies, but every once in a while I get the problem with the new movies as well.
I am going to do my best it trying to describe what is occurring.

Audio Sync Issues:
The problem here is that the audio sync is off. I can change the delay but the sync change throughout the entire movie. So I can't just use one value. The movies that I have this issue are:

Chain Reaction
XxX
A Few Good Men


Pauses
The video stops and starts a different sections throughout the entire video. Audio sync is not affected.

The Chronicles of Narnia : Prince Caspian


Artifacting
I get a banding row affect that is about 1 inch tall (on my 75" TV). Audio Sync is not affected.

Aquaman


When I encode I normally have anywhere from three to four computer encoding the movie. Two of the four computers are higher powered so I run to servers on each of the computers. The other two computers as lighter wait so I only run one server on them. My firewalls are off to make sure there are no issues in communicating. I use the CQ Mode with a CRF value of 20. I do not encode the audio from the Bluray, I just do a COPY Stream. IN the AviSynth settings, I leave everyting as default. I do remux the SUBs for subtitles.

I was wondering if there may be an issue on versions of the required programs. So perhaps that is the issue but not sure. Can you please advise what is the best version of the tools that I should install on all my computers?

Avisynth_260 or AviSynthPlus -r1576
FFDshow 20041012 or FFDshow_rev4532_2010717
JavaScript 8 Update 221 (build 1.8.0_221-b11)
MatroskaSplitter 23/06/2013


Any help is greatly appreciated. have a great day...

[Atak_Snajpera]

So lately I have been experiencing playback issue on some of my converted movies. Typically it is with older movies, but every once in a while I get the problem with the new movies as well.
I am going to do my best it trying to describe what is occurring.

Audio Sync Issues:
The problem here is that the audio sync is off. I can change the delay but the sync change throughout the entire movie. So I can't just use one value. The movies that I have this issue are:

Chain Reaction
XxX
A Few Good Men


Pauses
The video stops and starts a different sections throughout the entire video. Audio sync is not affected.

The Chronicles of Narnia : Prince Caspian


Artifacting
I get a banding row affect that is about 1 inch tall (on my 75" TV). Audio Sync is not affected.

Aquaman


When I encode I normally have anywhere from three to four computer encoding the movie. Two of the four computers are higher powered so I run to servers on each of the computers. The other two computers as lighter wait so I only run one server on them. My firewalls are off to make sure there are no issues in communicating. I use the CQ Mode with a CRF value of 20. I do not encode the audio from the Bluray, I just do a COPY Stream. IN the AviSynth settings, I leave everyting as default. I do remux the SUBs for subtitles.

I was wondering if there may be an issue on versions of the required programs. So perhaps that is the issue but not sure. Can you please advise what is the best version of the tools that I should install on all my computers?

Avisynth_260 or AviSynthPlus -r1576
FFDshow 20041012 or FFDshow_rev4532_2010717
JavaScript 8 Update 221 (build 1.8.0_221-b11)
MatroskaSplitter 23/06/2013


Any help is greatly appreciated. have a great day...

What exact movie release do you have?
Provide links from https://www.blu-ray.com/

[Viper714]

What exact movie release do you have?
Provide links from https://www.blu-ray.com/

Atak,
Here is the info all all of the movies. Hope this helps...
Chain Reaction = Link (https://www.blu-ray.com/movies/Chain-Reaction-Blu-ray/245/)
xXx = Link (https://www.blu-ray.com/movies/XXX-Blu-ray/70/)
A Few Good Men = Link (https://www.blu-ray.com/movies/A-Few-Good-Men-Blu-ray/351/)
The Chronicles of Narnia : Prince Caspian = Link (https://www.blu-ray.com/movies/The-Chronicles-of-Narnia-Prince-Caspian-Blu-ray/1189/)
Aquaman = Link (https://www.blu-ray.com/movies/Aquaman-Blu-ray/227124/)

As an added note, when I play the blu-ray disc from the computer (PowerDVD) everything works fine. When I play the AnyDvdHD rip with PowerDVD it plays fine. It seems to only happen after encoding...

[Atak_Snajpera]

Artifacting
I get a banding row affect that is about 1 inch tall (on my 75" TV). Audio Sync is not affected.
Can you take a photo of this banding? I'm curious how it looks like...

[Manul]

Atak,
Having problem starting EncodingServer. The window never appears but the process is present in Process Explorer. Any idea what could be in play? Please let me know if you need any further info. Thank you in advance. Sys info:
RipBot: 1.24.1
Operating System: Microsoft Windows 10 Pro 10.0.17763.615 (Win10 RS5 [1809] October 2018 Update)
DirectX: DirectX 12.0
CPU Type:10-Core Intel Core i9-9820X, 4000 MHz (40 x 100)
Motherboard Name: Asus Prime X299-Deluxe II
System Memory: 65212 MB (DDR4 SDRAM) 18-19-19-39 @ 4000 MHz
Video Adapter: TITAN X (Pascal) (12 GB)

[Viper714]

Can you take a photo of this banding? I'm curious how it looks like...

Atak,
I will do one better. Here is a link to an extract of the video. Please let me know if it works
LINK (https://www.dropbox.com/sh/35upcwk84qa2iud/AABTnOIVuSHrDFPcFELX_BCYa?dl=0)
You will see that artifacting. You can't miss it. It actually grays the screen out. I re-encoded the file a second time with CRF of 18 and that particular issue moved to a different area of the video. Like within 30 seconds of the issue you now see. There are other issues throughout the video as well...

[Viper714]

Atak,
Having problem starting EncodingServer. The window never appears but the process is present in Process Explorer. Any idea what could be in play? Please let me know if you need any further info. Thank you in advance.

I had the same problem months ago. Turns out that it was a startup program from my Logitech keyboard / Joysticks, "Logitech Gaming Framework." I disabled it from startup and the encoding server worked fine. Somewhere in this forum someone mentioned an issue with "Void Tools" program called "Everything," a really good file search program that is better than windows search. You might want to try to disable these two programs on startup if you have them and see if it works.
Worse case scenario, disable all the start up programs, restart the computer and see if the encoding server starts. If it does then enable the startup programs in sections until you find the culprit.
Hope that works. It did for me....

[Manul]

I had the same problem months ago. Turns out that it was a startup program from my Logitech keyboard / Joysticks, "Logitech Gaming Framework." I disabled it from startup and the encoding server worked fine. Somewhere in this forum someone mentioned an issue with "Void Tools" program called "Everything," a really good file search program that is better than windows search. You might want to try to disable these two programs on startup if you have them and see if it works.
Worse case scenario, disable all the start up programs, restart the computer and see if the encoding server starts. If it does then enable the startup programs in sections until you find the culprit.
Hope that works. It did for me....
You are da MAN! The moment I unloaded LGF server has started! Thank you for saving me a lot of time. Atak, any idea why it behaves like this? If it cannot be or not worth fixing it maybe put up some disclaimer? Anyway, thanks again!

[Atak_Snajpera]

Atak,
I will do one better. Here is a link to an extract of the video. Please let me know if it works
LINK (https://www.dropbox.com/sh/35upcwk84qa2iud/AABTnOIVuSHrDFPcFELX_BCYa?dl=0)
You will see that artifacting. You can't miss it. It actually grays the screen out. I re-encoded the file a second time with CRF of 18 and that particular issue moved to a different area of the video. Like within 30 seconds of the issue you now see. There are other issues throughout the video as well...

Will you be able to upload first 10 minutes (source video without audio) for me? You can use mkvtoolnix to split mkv.

It would be also helpful If you could do the same for other movies you mentioned earlier.

[Viper714]

You are da MAN! The moment I unloaded LGF server has started! Thank you for saving me a lot of time. Atak, any idea why it behaves like this? If it cannot be or not worth fixing it maybe put up some disclaimer? Anyway, thanks again!

No worries, glad I could help. :D
Just remember if you want to use the functions of your keyboard, you will have to enable it. I only disable it when I do encoding which is overnight.

[Viper714]

Will you be able to upload first 10 minutes (source video without audio) for me? You can use mkvtoolnix to split mkv.

It would be also helpful If you could do the same for other movies you mentioned earlier.

Sure I think I can do that. I will post when ready...
Thanks Atak, appreciate the help!
:thanks:

[byteshare]

You are da MAN! The moment I unloaded LGF server has started! Thank you for saving me a lot of time. Atak, any idea why it behaves like this? If it cannot be or not worth fixing it maybe put up some disclaimer? Anyway, thanks again!

No worries, glad I could help. :D
Just remember if you want to use the functions of your keyboard, you will have to enable it. I only disable it when I do encoding which is overnight.

If your hardware is compatible switch to Logitech G Hub: https://origin-gaming.logitech.com/en-us/microsite/ghub

I don't have those issues since switching.

[bluebeers]

You can use mkvtoolnix to split mkv.


is there a free tool that will let me see clips of the video while splitting MKVs?

also... I just did another test on my computer. I removed the 24gb of RAM and just put the original 4gb back. Why is there no difference in the amount of time it takes to encode the same video?

This is the 7-hour version of Godfather the movie that I'm encoding to HEVC x265 to make it much smaller. with 24gb of RAM, my PC takes 11 hours. with just 4gb of RAM, it takes the same 11 hours....

[Atak_Snajpera]

is there a free tool that will let me see clips of the video while splitting MKVs?

also... I just did another test on my computer. I removed the 24gb of RAM and just put the original 4gb back. Why is there no difference in the amount of time it takes to encode the same video?

This is the 7-hour version of Godfather the movie that I'm encoding to HEVC x265 to make it much smaller. with 24gb of RAM, my PC takes 11 hours. with just 4gb of RAM, it takes the same 11 hours....

Who did say that RAM does calculations?

[byteshare]

is there a free tool that will let me see clips of the video while splitting MKVs?

also... I just did another test on my computer. I removed the 24gb of RAM and just put the original 4gb back. Why is there no difference in the amount of time it takes to encode the same video?

This is the 7-hour version of Godfather the movie that I'm encoding to HEVC x265 to make it much smaller. with 24gb of RAM, my PC takes 11 hours. with just 4gb of RAM, it takes the same 11 hours....

You can use avidemux for spiting the file graphically: http://fixounet.free.fr/avidemux/download.html
MKVtoolnix lets you specify the time stamps but if you're aiming for a clip in the middle of a file it isn't as simple

As for RAM, more RAM won't make an encode go faster unless you're running of RAM for the amount you need to send to the CPU...meaning if you're encoding at a rate of 10MB/s (decoder to encoder) as long as the decoder can supply 10MB/s to RAM for the Encoder to process you're fine. Using 24GB of RAM just means your decoder is way ahead of the encoder, so 4GB likely means you are still way ahead.
Unless I'm wrong in how this works and in that case I'm sure Atak will correct me.

[Viper714]

Will you be able to upload first 10 minutes (source video without audio) for me? You can use mkvtoolnix to split mkv.

It would be also helpful If you could do the same for other movies you mentioned earlier.

Atak,
I've placed all of the extracted 15 minute video clips without audio (figured I would give a little more just in case) in the same area as the previous video LINK (https://www.dropbox.com/sh/35upcwk84qa2iud/AABTnOIVuSHrDFPcFELX_BCYa?dl=0). The Prince Caspian is actuall 33 minutes as the issue mentioned happens around the last 3 minutes of the video. Hope this helps. Please let me know what else you may need...
Thank you

[Viper714]

You can use avidemux for spiting the file graphically: http://fixounet.free.fr/avidemux/download.html
MKVtoolnix lets you specify the time stamps but if you're aiming for a clip in the middle of a file it isn't as simple

As for RAM, more RAM won't make an encode go faster unless you're running of RAM for the amount you need to send to the CPU...meaning if you're encoding at a rate of 10MB/s (decoder to encoder) as long as the decoder can supply 10MB/s to RAM for the Encoder to process you're fine. Using 24GB of RAM just means your decoder is way ahead of the encoder, so 4GB likely means you are still way ahead.
Unless I'm wrong in how this works and in that case I'm sure Atak will correct me.

I agree with Byteshare.
The amount of RAM does not change the speed of encoding. It is more CPU power. If you want to go faster then you need to use more PCs with Distributed Encoding.
I have four PCs and I can encode a 2 hour movie in about 30 - 45 minutes. It would be faster if two of my four PCs where higher CPU powered, but it is what you can afford as well.
Distributed Encoding is AWESOME!!! :D

[Atak_Snajpera]

Atak,
I've placed all of the extracted 15 minute video clips without audio (figured I would give a little more just in case) in the same area as the previous video LINK (https://www.dropbox.com/sh/35upcwk84qa2iud/AABTnOIVuSHrDFPcFELX_BCYa?dl=0). The Prince Caspian is actuall 33 minutes as the issue mentioned happens around the last 3 minutes of the video. Hope this helps. Please let me know what else you may need...
Thank you

But you have sent me RE-ENCODED files! I asked for original files which are normally saved in jobx folder (video.mkv) by EncodingClient.

[Viper714]

But you have sent me RE-ENCODED files! I asked for original files which are normally saved in jobx folder (video.mkv) by EncodingClient.

Oh, Okay I misunderstood. The jobx folder in the temp directory right? Will work on it...

[Viper714]

But you have sent me RE-ENCODED files! I asked for original files which are normally saved in jobx folder (video.mkv) by EncodingClient.

Okay Atak, All the files are uploaded in the same location. Please let me know if you need additional files....
Thanks again!

[Juha]

Hello, could you add an option to use VLC Media Player as an alternative video (or script) previewing tool? MPC does not work for me I just have a black screen that won't go away even though MPC says "playing". Maybe this is caused by a codec pack I use but I'm not sure. VLC has it's own codecs and filters so it would probably be more compatible. (I don't like VLC as my primary video player but as preview tool it would be fine).

[Viper714]

Hello, could you add an option to use VLC Media Player as a video (or script) previewing tool? MPC does not work for me I just have a black screen that won't go away even though MPC says "playing". Maybe this is caused by a codec pack I use but I'm not sure. VLC has it's own codecs and filters so it would probably be more compatible. (I don't like VLC as my primary video player but as preview tool it would be fine).

Juha,
I have a Media PC and I had to install codecs for it and my personal PC. What came with Windows wasn't enough. After researching many different packages on the web, I found a website Shark007. He efficiently bundled all the codecs in a easy install. You can find it HERE (https://shark007.net/index.html). What's really good about his package is that he regularly updates it. If you are going to use, I suggest using:

STANDARD Codecs
X64Components (if you have a 64-bit system)


Regarding VLC, I also use it. It is a great program. Just make user you have the latest version. Here is an article that came out a day or so ago about potential vulnerability LINK (https://www.ghacks.net/2019/07/24/confusion-critical-vlc-media-player-vulnerability/).
Hope that helps

[byteshare]

Hello, could you add an option to use VLC Media Player as an alternative video (or script) previewing tool? MPC does not work for me I just have a black screen that won't go away even though MPC says "playing". Maybe this is caused by a codec pack I use but I'm not sure. VLC has it's own codecs and filters so it would probably be more compatible. (I don't like VLC as my primary video player but as preview tool it would be fine).
I have RipBot on 4 PCs and all have the same version of K-Lite Codec Pack and I'm not sure what I've done differently on each machine because on 2 of the 4 I have the same issue you are having. All are Win10 either Pro or Home but 1 Pro works and 1 Home works.
If I need to see the error of a AVISynth script I've been using AVSpmod x64bit from here: https://github.com/nekopanda/AvsPmod/releases
I hadn't posted about it because I just figured it would be a mess to figure out and I had a work around.

[BLKMGK]

Any ideas on what is causing this (I'm using DE)?

Encoding started...
""\\XXXXX\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\XXXXX\RipBot264temp\job883\Chunks\5.avs" -strict -1 -f yuv4mpegpipe - | "\\XXXXXXX\Ripbot264temp\tools\x265\x265_x64.exe" --seek 4 --colorprim bt709 --transfer bt709 --colormatrix bt709 --crf 18 --fps 24000/1001 --min-keyint 24 --keyint 240 --frames 1441 --sar 1:1 --preset slow --ctu 32 --y4m --pools "+" --output "\\XXXXXXX\RipBot264temp\job883\Chunks\5.265" -"
x265 [error]: unable to open input file <->


I have a number of Windows clients. For some reason TV shows NEVER show this error but movies often (but not always) do. It looks like a permissions issue but the share is wide open and if I type in the address on the client machine I get right in and appear to have Full Control as intended. I'm at wit's end and have been unable to find a correlation to understand why this doesn't happen on everything. I feel like it must be something silly but cannot find it and have been trying for at least a month.

This slow my encoding greatly to say the least when it occurs :( Help please!

P.S. When it occurs no client except the host is able to encode.

[Atak_Snajpera]

What happens if you try to open 5.avs in mpc-hc player? (from server side)

[Juha]

I tried Shark codec pack but I had issues with it so I removed it.

I however fixed the problem I had with preview function in Ripbot. It was indeed a codec issue. I had K-Lite Codec Pack (full) which I uninstalled and then re-installed using an option to install codecs only for the Media Player Classic HC (which I use as my primary video player).

[slalom]

So lately I have been experiencing playback issue on some of my converted movies. Typically it is with older movies, but every once in a while I get the problem with the new movies as well.
I am going to do my best it trying to describe what is occurring.

Audio Sync Issues:
The problem here is that the audio sync is off. I can change the delay but the sync change throughout the entire movie. So I can't just use one value. The movies that I have this issue are:

Chain Reaction
XxX
A Few Good Men

Try this
Open Avisynth, in output speed select assume -> x fps
(fps of the movie)

[byteshare]

I tried Shark codec pack but I had issues with it so I removed it.

I however fixed the problem I had with preview function in Ripbot. It was indeed a codec issue. I had K-Lite Codec Pack (full) which I uninstalled and then re-installed using an option to install codecs only for the Media Player Classic HC (which I use as my primary video player).
So I first tried on two computers (A working, B not working):
1. reinstalled KLite Codec Mega pack with all defaults, A still working, B not working
2. For B reinstalled KLite Codec Mega pack all defaults but checked "Install codecs for MPC-HC only", had to install an external Haali Media Splitter, same issue of MPC still not giving me a proper preview when avisynth has an issue (Cannot render the file).

Glad you found a fix for you, not sure what is causing my issue :(

[Juha]

So I first tried on two computers (A working, B not working):
1. reinstalled KLite Codec Mega pack with all defaults, A still working, B not working
2. For B reinstalled KLite Codec Mega pack all defaults but checked "Install codecs for MPC-HC only", had to install an external Haali Media Splitter, same issue of MPC still not giving me a proper preview when avisynth has an issue (Cannot render the file).

Glad you found a fix for you, not sure what is causing my issue :(

Haali is needed by Ripbot but not by K-lite Codec pack. First install Ripbot and Haali and then remove and install K-lite again. K-lite installer should prompt you to uninstall Haali, for which you should say no as it's needed by Ripbot. Then K-lite will be installed and everything works fine this way at least for me. If this does not help, try K-lite Full pack instead of Mega pack. You probably don't need the Mega pack. Here is the list of differences between various K-lite editions: https://www.codecguide.com/download_kl.htm

[byteshare]

Haali is needed by Ripbot but not by K-lite Codec pack. First install Ripbot and Haali and then remove and install K-lite again. K-lite installer should prompt you to uninstall Haali, for which you should say no as it's needed by Ripbot. Then K-lite will be installed and everything works fine this way at least for me. If this does not help, try K-lite Full pack instead of Mega pack. You probably don't need the Mega pack. Here is the list of differences between various K-lite editions: https://www.codecguide.com/download_kl.htm
So I tried:
1 Install Haali Media Splitter
2 Remove & reinstall K-lite.
3 K-lite installer should prompt you to uninstall Haali, for which you should say no.
--And then the same with with K-lite Full pack instead of Mega pack.

For testing I tried uninstalling K-lite, rebooting, then only installing ffdshow and matroska splitter for RipBot and still get the same issue.
I am using the same files on different machines...so maybe it has something to do with my video card? IDK so for now if I need to test I just copy the RipBot script into AvsPmod.

[Juha]

When you uninstalled K-lite, it asked you to give feedback. What you chose? I answered I don't need it anymore. Sometimes uninstallers ask why are you removing it so that it knows if it should remove all the settings completely or just some of them in case of soon-to-happen re-installation. I recommend trying once more and choosing I don't need it anymore. Then I would try again if Ripbot preview works without K-lite being installed.

I have 2 machines and both had that preview issue and it got fixed on both.

If the preview still does not work for you despite K-lite is uninstalled it's hard to say what could be the issue. Maybe you have some codec remnants somewhere? In this case I would uninstall everything codec related stuff I could and then disable every codec that is left by using this tool: http://www.nirsoft.net/utils/installed_codec.html (make a system restore point in case of problems) and then install back the stuff that Ripbot is needing. Then try again the preview.

Have you also updated your graphics drivers?

[nekrosoft13]

Atak,

In administrator mode, only ripbot264 (Without EncodingServer.exe started due to HT mode)

https://image.noelshack.com/fichiers/2019/29/2/1563309377-2019-07-16-22h34-49.png

https://image.noelshack.com/fichiers/2019/29/2/1563309411-2019-07-16-22h36-38.png

the log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

If i make only EncodingServer.exe /node 0 (via cmd.exe) EncodingServer.exe doesn't run

https://image.noelshack.com/fichiers/2019/29/2/1563310074-2019-07-16-22h47-32.png

However, if i do EncodingServer.exe /affinity 1 - EncodingServer.exe start

https://image.noelshack.com/fichiers/2019/29/2/1563310253-2019-07-16-22h49-57.png

https://image.noelshack.com/fichiers/2019/29/2/1563310363-2019-07-16-22h50-04.png

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=1
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

And now, as EncodingServer.exe run, if i do EncodingServer.exe /node 0 or EncodingServer.exe /node 1, nothing happens either

https://image.noelshack.com/fichiers/2019/29/2/1563311228-2019-07-16-23h06-45.png

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

Nevertheless, i don't know if it's important but i noticed that when i execute EncodingServer.exe /node 0

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=1
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

And if i do EncodingServer.exe /node 1

The log is:

ProcessAffinityMask=111111111111111111111111111111111111
SystemAffinityMask=111111111111111111111111111111111111
Node=0
ProcessAffinityMaskPerNode=111111111111111111111111111111111111
SystemAffinityMaskPerNode=111111111111111111111111111111111111

https://image.noelshack.com/fichiers/2019/29/2/1563312130-2019-07-16-23h21-47.png

it's seems that the node number is switched when we do EncodingServer.exe /node 1 or EncodingServer.exe /node 0 twice

https://image.noelshack.com/fichiers/2019/29/2/1563312193-2019-07-16-23h22-54.png

Bye

did you get it to work? I been having the smae issue for few months.

[Atak_Snajpera]

did you get it to work? I been having the smae issue for few months.

Try this https://www.mediafire.com/file/aw8gqz7snt7twlb/EncodingServer.exe/file

[byteshare]

When you uninstalled K-lite, it asked you to give feedback. What you chose? I answered I don't need it anymore. Sometimes uninstallers ask why are you removing it so that it knows if it should remove all the settings completely or just some of them in case of soon-to-happen re-installation. I recommend trying once more and choosing I don't need it anymore. Then I would try again if Ripbot preview works without K-lite being installed.

I have 2 machines and both had that preview issue and it got fixed on both.

If the preview still does not work for you despite K-lite is uninstalled it's hard to say what could be the issue. Maybe you have some codec remnants somewhere? In this case I would uninstall everything codec related stuff I could and then disable every codec that is left by using this tool: http://www.nirsoft.net/utils/installed_codec.html (make a system restore point in case of problems) and then install back the stuff that Ripbot is needing. Then try again the preview.

Have you also updated your graphics drivers?
When I uninstalled Klite I choose I wanted a fresh install.
I have updated my drivers from time to time but I don't recall the preview ever working for the PCs that it isn't working on.
I haven't tried that tool, but I think avspmod is enough for now, since I've already spent a bit of time on this.
An extra step but it works and I don't need to use the preview very often except when something goes wrong or trying a new avisynth script.
Thanks for the help. Next time I format my computer I'll see what happens, or if I get annoyed enough I'll come back to this, lol.

[nekrosoft13]

Try this https://www.mediafire.com/file/aw8gqz7snt7twlb/EncodingServer.exe/file

that worked ! thank you :thanks:

I have both nodes at 100%, will post a screenshot later.

[gabbett1]

Hello, Atak. I have distributed encoding working, but for some reason the host computer will not encode, only the secondary PC's do the work until it's time to mux. Any thoughts? What info can I get you to help figure this out? Thanks.

[Atak_Snajpera]

Screenshots ,please.

[nekrosoft13]

Here you go

http://oi65.tinypic.com/2eq70rd.jpg

[Atak_Snajpera]

Here you go

http://oi65.tinypic.com/2eq70rd.jpg

LoL! It was for gabbett1 ;)

BTW. I would add two extra servers because CPU usage is still not at constant 100%.

[gabbett1]

Screenshots ,please.

LOL, figures. It must have updated or something. It's working fine. disregard. Thanks.

[Atak_Snajpera]

v1.25.0
Added: Variable priority mode to EncodingServer.
In this mode EncodingClient will be automatically changing encoding priority in order to balance progress on multiple servers running on the same machine.
Added: EncodingClient now also shows encoding priority set on server side and selected encoding numa node.
Fixed: EncodingClient is now copying file to shared folder in buffered mode.
Windows disables buffered write if write buffer in application is higher than 16MiB.
Because of this write buffer has been reduced from 64MiB to 16MiB.
Buffered write significantly reduces copy time on machine with huge amount of RAM.
Changed: lsmash is now a default decoder for all formats except VC1. FFMS will be still used for VC1.
Fixed: EncodingServer not starting on some Xeon machines with high number of threads.
Fixed: few minor bugs found since last version.


Progress balancing OFF
https://i.imgsafe.org/32/322ed275a6.png

Progress balancing ON
https://i.imgsafe.org/32/322ed0e740.png

Important message
Use variable priority only if:
1) you encode on single machine (for example with high number of cores/threads)
2) ...and use multiple servers to get full cpu usage
3) ...and chunks size is set to 0 (one big chunk for each active server)

Otherwise encoding time will be bigger than in default mode.

[byteshare]

v1.25.0
Added: Variable priority mode to EncodingServer.
In this mode EncodingClient will be automatically changing encoding priority in order to balance progress on multiple servers running on the same machine.
Added: EncodingClient now also shows encoding priority set on server side and selected encoding numa node.
Fixed: EncodingClient is now copying file to shared folder in buffered mode.
Windows disables buffered write if write buffer in application is higher than 16MiB.
Because of this write buffer has been reduced from 64MiB to 16MiB.
Buffered write significantly reduces copy time on machine with huge amount of RAM.
Changed: lsmash is now a default decoder for all formats except VC1. FFMS will be still used for VC1.
Fixed: EncodingServer not starting on some Xeon machines with high number of threads.
Fixed: few minor bugs found since last version.

Awesome! Thank you.
I'm seeing *.lwi files now, is that from the lsmash?

[byteshare]

I'm using this custom script in the custom area:
video=video.ConvertBits(16).ConvertToStacked().SMDegrain(tr=3,thSAD=400,thSADC=150,refinemotion=false,contrasharp=true,plane=4,pel=2,prefilter=4,Truemotion=False,chroma=true,hpad=32,vpad=32,str=2,amp=1,lsb_in=true,lsb=true,lsb_out=true).ConvertFromStacked().ConvertBits(8).FineDehalo().FastLineDarkenMOD4().LSFmod()

The AVISynth Preview doesn't throw an error and it also works fine in AVSpmod.
On the Encoding server I either see it get to encoding of x265 and then crash (see below):
1|153422453|ENCODING_STARTED
2|153422750|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=13;RAM=40;DECODER=0;ENCODER=0;OTHER=13;ENCODING_PRIORITY=low;
3|153423078|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=13;RAM=40;DECODER=0;ENCODER=0;OTHER=13;ENCODING_PRIORITY=low;
4|153423421|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=13;RAM=40;DECODER=0;ENCODER=0;OTHER=13;ENCODING_PRIORITY=low;
5|153423890|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=18;RAM=40;DECODER=2;ENCODER=0;OTHER=16;ENCODING_PRIORITY=low;
6|153424265|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=18;RAM=40;DECODER=2;ENCODER=0;OTHER=16;ENCODING_PRIORITY=low;
7|153424625|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=18;RAM=40;DECODER=2;ENCODER=0;OTHER=16;ENCODING_PRIORITY=low;
8|153425125|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=28;RAM=40;DECODER=6;ENCODER=0;OTHER=22;ENCODING_PRIORITY=low;
9|153425500|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=28;RAM=40;DECODER=6;ENCODER=0;OTHER=22;ENCODING_PRIORITY=low;
10|153425859|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=28;RAM=40;DECODER=6;ENCODER=0;OTHER=22;ENCODING_PRIORITY=low;
11|153426359|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=40;DECODER=6;ENCODER=0;OTHER=21;ENCODING_PRIORITY=low;
12|153426734|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=40;DECODER=6;ENCODER=0;OTHER=21;ENCODING_PRIORITY=low;
13|153427093|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=40;DECODER=6;ENCODER=0;OTHER=21;ENCODING_PRIORITY=low;
14|153427609|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=22;RAM=42;DECODER=3;ENCODER=0;OTHER=19;ENCODING_PRIORITY=low;
15|153427984|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=22;RAM=42;DECODER=3;ENCODER=0;OTHER=19;ENCODING_PRIORITY=low;
16|153428375|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=22;RAM=42;DECODER=3;ENCODER=0;OTHER=19;ENCODING_PRIORITY=low;
17|153428921|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=46;DECODER=4;ENCODER=0;OTHER=23;ENCODING_PRIORITY=low;
18|153429328|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=46;DECODER=4;ENCODER=0;OTHER=23;ENCODING_PRIORITY=low;
19|153429765|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=46;DECODER=4;ENCODER=0;OTHER=23;ENCODING_PRIORITY=low;
20|153430390|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=72;RAM=47;DECODER=16;ENCODER=0;OTHER=56;ENCODING_PRIORITY=low;
21|153430718|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ERROR;CPU=72;RAM=47;DECODER=16;ENCODER=0;OTHER=56;ENCODING_PRIORITY=low;
22|153430984|SERVER_IDLE
23|153431500|ENCODING_STARTED
Or I see an error in the main (not TCP Communication):
x265 [error]: unable to open input file <->

Even if I close the Encoding Server and restart that and no matter how many times the Encoding server retries a chunk it doesn't seem to work but if I close RipBot and try again it works?

Like I said it works but wasn't sure if there was something more "interesting" going on.

[BLKMGK]

Any ideas on what is causing this (I'm using DE)?

Encoding started...
""\\XXXXX\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\XXXXX\RipBot264temp\job883\Chunks\5.avs" -strict -1 -f yuv4mpegpipe - | "\\XXXXXXX\Ripbot264temp\tools\x265\x265_x64.exe" --seek 4 --colorprim bt709 --transfer bt709 --colormatrix bt709 --crf 18 --fps 24000/1001 --min-keyint 24 --keyint 240 --frames 1441 --sar 1:1 --preset slow --ctu 32 --y4m --pools "+" --output "\\XXXXXXX\RipBot264temp\job883\Chunks\5.265" -"
x265 [error]: unable to open input file <->


I have a number of Windows clients. For some reason TV shows NEVER show this error but movies often (but not always) do. It looks like a permissions issue but the share is wide open and if I type in the address on the client machine I get right in and appear to have Full Control as intended. I'm at wit's end and have been unable to find a correlation to understand why this doesn't happen on everything. I feel like it must be something silly but cannot find it and have been trying for at least a month.

This slow my encoding greatly to say the least when it occurs :( Help please!

P.S. When it occurs no client except the host is able to encode.

Still stuck on this! It appears to only occur on jobs for which I'm running AVISynth filtering. I'd really appreciate some input on what to look for as this filtering is really something I need, thanks!

[Atak_Snajpera]

Show us your avisynth script.

[BLKMGK]

Show us your avisynth script.

I'm clicking on the AVIsynth button, selecting the DeNoise dropd-own, and then selecting the bottom-most filter. I then either leave it at zero or bump the level to 2 depending upon the video source. I also allow it to autocrop.

Nothing custom. It acts as if there's a file not being shared and it cannot access it. If I do videos onesy twosey it doesn't seem to happen as often but if I stack up a queue of say 9 of them I start running into this. I have managed to process videos with these filters enabled but when this occurs it always seems to be on filtered videos.

The share is set to allow "everybody" full control and I can use the filemanager on the problem systems to get to the source files but running the CMD in the job folder results in the same error, I'm not sure what it cannot access. I have a stack of about 4 videos doing it right now but I also just added some things that are much shorter and lower resolution - they will process just fine I'm sure. I'm now on 1.25 on most of my machines but my Win7 cannot run that encoder Server for some reason and is on an older one. A brand new updated Win10 machine is having this issue too though.

[Atak_Snajpera]

I'm clicking on the AVIsynth button, selecting the DeNoise dropd-own, and then selecting the bottom-most filter. I then either leave it at zero or bump the level to 2 depending upon the video source. I also allow it to autocrop.

Nothing custom. It acts as if there's a file not being shared and it cannot access it. If I do videos onesy twosey it doesn't seem to happen as often but if I stack up a queue of say 9 of them I start running into this. I have managed to process videos with these filters enabled but when this occurs it always seems to be on filtered videos.

The share is set to allow "everybody" full control and I can use the filemanager on the problem systems to get to the source files but running the CMD in the job folder results in the same error, I'm not sure what it cannot access. I have a stack of about 4 videos doing it right now but I also just added some things that are much shorter and lower resolution - they will process just fine I'm sure. I'm now on 1.25 on most of my machines but my Win7 cannot run that encoder Server for some reason and is on an older one. A brand new updated Win10 machine is having this issue too though.

It looks like you are using GPU filter KNLMeansCL for denoising. I suppose that filter is having some issues accessing your GPU. Perhaps device id is not set correctly. I'm just guessing now...

Run EncodingServer.exe on each machine (host as well) and take a screenshot.

[BLKMGK]

Host both idle and encoding at 5ghz

https://i.postimg.cc/VL4cdXD5/host-encoding.jpg
https://i.postimg.cc/fbf7NTw4/host-idle.jpg

Second machine - Ryzen 3900X and yes it's blowing away the other machine when it encodes! :-O

https://i.postimg.cc/J0MhZJv0/ryzen.png
https://i.postimg.cc/7hHxf4JT/ryzen2.png
https://i.postimg.cc/g0hRKVSH/ryzen3.png

I hadn't realized this leveraged the GPU and it hadn't been set on the Ryzen machine or my others that I've not posted so as not to flood the site with images. I did set the GPU on the second box (it's crappy but usually headless) but same error currently. If the GPU is the issue, is there a CPU based filter that's as good? IMO this one removes film grain and some glitches amazingly well! It may be that my GPU in the second machine is not good enough perhaps? I've checked another machine with just an Intel GPU and no joy there either.

Please let me know if I can provide anything else to figure this one out, I really appreciate the assistance.:thanks: I'd be happy to switch to a CPU based filter if I can maintain quality.

I'll try hosting an encode on my Ryzen box too to see if it encounters this problem but cannot do it this moment.

[Atak_Snajpera]

If you are going to use KNLMeansCL in DE mode then you must:
1) have decent gpus in all PC's. GeForce 210 is just a crap GPU (Supports up to OpenCL 1.1 [knlmeansCL requires 1.2] and only two compute units!).
2) manually set device id in encoding server if you have more than one GPU. One of your machines has Intel GPU and GeForce 1070 for example.
This will tell encodingserver to use GeForce 1070 (faster)
EncodingServer.exe /knlmeanscl-opencl-device-id 1

In your case It might be a good idea to run one server using Intel GPU and other GeForce 1070 .
EncodingServer.exe /knlmeanscl-opencl-device-id 0
EncodingServer.exe /knlmeanscl-opencl-device-id 1

This way you will be using literally everything what can compute ;) CPU+iGPU+dGPU

If you want you can also try running KNLMeansCL on CPU but be warned it will be veeeeeeeerrrrrrryyyyy slooooooooow.
EncodingServer.exe /knlmeanscl-opencl-device-type CPU

[BLKMGK]

If you are going to use KNLMeansCL in DE mode then you must:
1) have decent gpus in all PC's. GeForce 210 is just a crap GPU (Supports up to OpenCL 1.1 [knlmeansCL requires 1.2] and only two compute units!).
2) manually set device id in encoding server if you have more than one GPU. One of your machines has Intel GPU and GeForce 1070 for example.
This will tell encodingserver to use GeForce 1070 (faster)
EncodingServer.exe /knlmeanscl-opencl-device-id 1

In your case It might be a good idea to run one server using Intel GPU and other GeForce 1070 .
EncodingServer.exe /knlmeanscl-opencl-device-id 0
EncodingServer.exe /knlmeanscl-opencl-device-id 1

This way you will be using literally everything what can compute ;) CPU+iGPU+dGPU

If you want you can also try running KNLMeansCL on CPU but be warned it will be veeeeeeeerrrrrrryyyyy slooooooooow.
EncodingServer.exe /knlmeanscl-opencl-device-type CPU

Thank you, that's a HUGE help! What's the minimum GPU you'd recommend? I assume a 1050 would work? I have one of those in a Linux machine for realtime streaming, the 210 was bought simply to get me video on the Ryzen box but I'll upgrade it. I wish I could interleave jobs and send the ones that don't require GPU to just those machines lol. The Ryzen box does require two encoding servers to get full utilization BTW, it's close to triple the speed of my 8700K :eek: I'll see how it handles the filter on the CPU :devil:

P.S. what are these *.lwi files now showing up with the new client? Seem to contain data about the various videos.

[Atak_Snajpera]

*.lwi is an index file for video file. It is required to have frame accurate seeking.

[BLKMGK]

Must require something other than OpenCL 1.2, I've got a GTX 680 that claims to support OpenCL 1.2 with a whopping 4 compute units that also hits this error - argh! Looks like I might need more than one new video card! :O

[Atak_Snajpera]

Try with 1050

[byteshare]

*.lwi is an index file for video file. It is required to have frame accurate seeking.
Thank you for clarifying that.

[userx]

Hello,
For some time the first pass in 2-pass mode won't start at my remote machine. 2nd pass works like expected. If I start a job on commandline manually I get following output:
PS Microsoft.PowerShell.Core\FileSystem::\\USERX-MASTER\RipBot264temp\job5\Chunks> .\39.cmd
"\\HOST\RipBot264temp\job5\Chunks"
CMD.EXE wurde mit dem oben angegebenen Pfad als aktuellem Verzeichnis gestartet.
UNC-Pfade werden nicht unterstützt.
Stattdessen wird das Windows-Verzeichnis als aktuelles Verzeichnis gesetzt.
""\\HOST\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\HOST\RipBot264temp\job5\Chunks\39.avs" -strict -1 -f yuv4mpegpipe - | "\\HOST\Ripbot264temp\tools\x264\x264_x64.exe" --seek 48 --colorprim bt709 --transfer bt709 --colormatrix bt709 --opencl --opencl-device 0 --opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead.clbin" --pass 1 --bitrate 3000 --stats "\\HOST\RipBot264temp\job5\Chunks\39.stats" --fps 25 --force-cfr --min-keyint 25 --keyint 250 --frames 1488 --sar 1:1 --level 4.1 --aud --nal-hrd vbr --vbv-bufsize 25000 --vbv-maxrate 25000 --b-pyramid none --ref 5 --stdin y4m --output NUL -"
y4m [info]: 1280x720p 1:1 @ 25/1 fps (cfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x264 [info]: OpenCL acceleration enabled with Advanced Micro Devices, Inc. Capeverde (SI)
x264 [info]: profile Main, level 4.1, 4:2:0, 8-bit
A s s e r t i o n f a i l e d !

P r o g r a m : \ \ H O S T \ R i p b o t 2 6 4 t e m p \ t o o l s \ x 2 6 4 \ x 2 6 4 _ x 6 4 . e x e
F i l e : e n c o d e r / s l i c e t y p e . c , L i n e 1 9 8 9

E x p r e s s i o n : c o s t > = 0


Can someone give me a hint how i can solve this problem?
Thanks

[Atak_Snajpera]

Are you sure you are using latest version?

I'm asking because this looks odd
--opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead.clbin"
It should be with digit related with server instance
--opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead_1.clbin"

[userx]

Are you sure you are using latest version?

I'm asking because this looks odd

It should be with digit related with server instance

yes, it's the latest version 1.25
in ...\AppData\Local\Temp\ i found
x264_lookahead.clbin
x264_lookahead_1.clbin
x264_lookahead_2.clbin
x264_lookahead_3.clbin
All with file changed date from yesterday

Does it matter if i start Ripbot264.exe on remote machine which start EncodingServer instances automatically compared to starting EncodingServer instances alone?

[Atak_Snajpera]

Can you upload all .cmd files from Chunks folder?
Are you using multiple servers on the same machine? If yes then does this problem also occur if you use only one?

[brumsky]

Hey Atak_Snajpera,

Would it be possible to configure a way to use analysis save and load while doing distributed encoding? Thinking through it you'd have to code a way to save a separate analysis file for each chuck and configure a way to reuse them.

thanks for all your hard work.

[Atak_Snajpera]

But why would you need that?

[brumsky]

But why would you need that?

When I'm trying to find the ideal bitrate or if I don't like the quality of the encode. Really, just about any time you have to redo the encode. Some settings can be tweaked like subme or aq-mode. I have encode that have a dark scene that don't look good and I'll change aq-mode to 3 and it looks a lot better.

I know other settings like --tu-inter can't be changed otherwise it'll affect the data stored in the analysis file.

Just a thought, if it's to much work I understand.

[byteshare]

Two questions about the LWI files:
1. Once they get made if you keep the file and later add a video (new job) that has a *.lwi file will it be reused or does a new one get generated?
2. Is it possible to have the .lwi files written to the job folder?
--So that clean up is easier and to save a little bit of time in the cases (like myself) when the RipBot temp folder is on a different drive, such as video on a physical drive and the temp folder on a SSD.

[Atak_Snajpera]

Unfortunatelly lsmash plugin does not have an option to select folder for index file. This annoys me as well...

[slalom]

On the latest version, some jobs after creating chunks, don't start

[byteshare]

Unfortunatelly lsmash plugin does not have an option to select folder for index file. This annoys me as well...
Thank you for letting me know, and glad I'm not the only one annoyed, LOL

On the latest version, some jobs after creating chunks, don't start
Does the encoding server start and stop?
Or do the chunks get made and the chunk list shows and then nothing happens after that?
Any logs?

I had an issue with starting a job, checking a few chunks but not liking the output so I changed the job, but then the first few chunks weren't over written and the job would finish but then started all over again. It was strange. I manually (right click job > clear) cleared the DE progress and it worked. So, anything like that?

[userx]

Can you upload all .cmd files from Chunks folder?
Are you using multiple servers on the same machine? If yes then does this problem also occur if you use only one?

Chunks_cmds (https://www.dropbox.com/s/ox3a5lbs698exbe/Chunks.rar?dl=0)

I usually using multiple servers but for testing i started an single instance which led to the same result.

output for this job on remote machine:
Y:\job6\Chunks>1.cmd
""\\USERX-MASTER\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\USERX-MASTER\RipBot264temp\job6\Chunks\1.avs" -strict -1 -f yuv4mpegpipe - | "\\USERX-MASTER\Ripbot264temp\tools\x264\x264_x64.exe" --seek 0 --colorprim bt709 --transfer bt709 --colormatrix bt709 --opencl --opencl-device 0 --opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead_2.clbin" --pass 1 --bitrate 3000 --stats "\\USERX-MASTER\RipBot264temp\job6\Chunks\1.stats" --fps 25 --force-cfr --min-keyint 25 --keyint 250 --frames 1500 --sar 1:1 --level 4.1 --aud --nal-hrd vbr --vbv-bufsize 25000 --vbv-maxrate 25000 --b-pyramid none --ref 5 --stdin y4m --output NUL -"
y4m [info]: 1280x720p 1:1 @ 25/1 fps (cfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x264 [info]: OpenCL acceleration enabled with Advanced Micro Devices, Inc. Capeverde (SI)
x264 [info]: profile Main, level 4.1, 4:2:0, 8-bit
A s s e r t i o n f a i l e d !

P r o g r a m : \ \ U S E R X - M A S T E R \ R i p b o t 2 6 4 t e m p \ t o o l s \ x 2 6 4 \ x 2 6 4 _ x 6 4 . e x e
F i l e : e n c o d e r / s l i c e t y p e . c , L i n e 1 9 8 9

E x p r e s s i o n : c o s t > = 0

[LigH]

The location of the error and its description appear to me like a miscalculation in the x264 encoder; I wonder if the reason may be the OpenCL acceleration processing wrong data, maybe due to task switching... bold guess. :o
__

Unfortunatelly lsmash plugin does not have an option to select folder for index file. This annoys me as well...

HolyWu introduced a 'cachefile' parameter. (https://forum.doom9.org/showthread.php?p=1881332#post1881332)

[slalom]

Does the encoding server start and stop?
Or do the chunks get made and the chunk list shows and then nothing happens after that?
Any logs?

I had an issue with starting a job, checking a few chunks but not liking the output so I changed the job, but then the first few chunks weren't over written and the job would finish but then started all over again. It was strange. I manually (right click job > clear) cleared the DE progress and it worked. So, anything like that?
Servers are off. I can start them but nothing happens
Yes
Show log doesn't show anything

Tried that, even removed the job and added it again

[Atak_Snajpera]

The location of the error and its description appear to me like a miscalculation in the x264 encoder; I wonder if the reason may be the OpenCL acceleration processing wrong data, maybe due to task switching... bold guess. :o
__



HolyWu introduced a 'cachefile' parameter. (https://forum.doom9.org/showthread.php?p=1881332#post1881332)
I think it Has to be some driver issue. Capeverde is a first generation of gcn (HD 7000 series) with only 1gib of vram.

[duffbeer]

Can I ask why lsmash is now the default decoder? What is the advantage?

I only ask because preparing the jobs now seems much slower. Cropping takes much longer and it seems to take too long each time I refresh the preview frame (to find a bright coloured frame for cropping accurately). Is this due to lsmash?

[Atak_Snajpera]

Can I ask why lsmash is now the default decoder? What is the advantage?

I only ask because preparing the jobs now seems much slower. Cropping takes much longer and it seems to take too long each time I refresh the preview frame (to find a bright coloured frame for cropping accurately). Is this due to lsmash?

Because of this
https://forum.doom9.org/showthread.php?p=1879893#post1879893

lsmash also has better support for interlaced AVC (does not duplicate frame count).

[duffbeer]

Because of this
https://forum.doom9.org/showthread.php?p=1879893#post1879893

lsmash also has better support for interlaced AVC (does not duplicate frame count).

OK. Seems odd as I never had any problems with the previous decoder.

Does this change explain the speed problems I mentioned?
I may have to go back to 1.24 as the new version is much slower now.

[Atak_Snajpera]

Lsmash tries to be more accurate while seeking.

[SKPN]

Is there a way to set the aspect ratio separate from the resolution?

[rayers1060]

Hi, is is possible to modify the encoding server to work on a separate network?. specifically I have a few idle pc's that are offsite and would like to utilize them for encoding. We dont want to create a VPN. Is this a request that could be in the future?. RipBot264 works great with the few pc's that we have on the same Lan ??, a few more would definitely help

[slalom]

Is there a way to set the aspect ratio separate from the resolution?
Have you tried custom setting for resolution?

[SKPN]

Have you tried custom setting for resolution?

Yea, but there is no option to set the aspect ratio when doing that.

[Atak_Snajpera]

Yea, but there is no option to set the aspect ratio when doing that.

You can specify aspect ratio in encoding profile. Just add --sar with proper aspect ratio

--sar <width:height|int> Sample Aspect Ratio, the ratio of width to height of an individual pixel.
Choose from 0=undef, 1=1:1("square"), 2=12:11, 3=10:11, 4=16:11,
5=40:33, 6=24:11, 7=20:11, 8=32:11, 9=80:33, 10=18:11, 11=15:11,
12=64:33, 13=160:99, 14=4:3, 15=3:2, 16=2:1 or custom ratio of <int:int>. Default 0

[byteshare]

Hi, is is possible to modify the encoding server to work on a separate network?. specifically I have a few idle pc's that are offsite and would like to utilize them for encoding. We dont want to create a VPN. Is this a request that could be in the future?. RipBot264 works great with the few pc's that we have on the same Lan ??, a few more would definitely help
You can do it with a VPN for sure, but if you don't want a VPN have you tried setting public IP addresses for the servers?
That would seem like a security risk IMO.

[ReinerSchweinlin]

Hi, is is possible to modify the encoding server to work on a separate network?. specifically I have a few idle pc's that are offsite and would like to utilize them for encoding. We dont want to create a VPN. Is this a request that could be in the future?. RipBot264 works great with the few pc's that we have on the same Lan ??, a few more would definitely help
That´s exactly what I do now, it already works with ripbot.
If you have stable connections with static IP, you´re all set.
Home-DSL or similar resets it´s IP adress within 24h in most countries, so you´d have to whip up some dyndns stuff... If the connection brakes down while encoding, in 80% the encode gets stuck, so you have to monitor everything alle the time.
For VPN, you could use almost anything you can think of, ripbot doesn´t care.... I tried hamachi, AVM-Fritzbox VPN, etc....

[slalom]

Does the encoding server start and stop?
Or do the chunks get made and the chunk list shows and then nothing happens after that?
Any logs?

I had an issue with starting a job, checking a few chunks but not liking the output so I changed the job, but then the first few chunks weren't over written and the job would finish but then started all over again. It was strange. I manually (right click job > clear) cleared the DE progress and it worked. So, anything like that?
Removed the audio and added the file
Job starts to encode

I have three other files to "mute" and add them too

I'll do my job eventually

[byteshare]

Removed the audio and added the file
Job starts to encode

I have three other files to "mute" and add them too

I'll do my job eventually

I have that issue from time to time where I need to strip the file of everything but the video or it doesn't add to RipBot.

[rayers1060]

You can do it with a VPN for sure, but if you don't want a VPN have you tried setting public IP addresses for the servers?
That would seem like a security risk IMO.

I must be missing something... The easy part is getting Encoding Client to connect with the remote server (I simply add static address and the port to Distributed Encoding server list) - remote encoding server reports "Client established connection with server" so all is good connecting to server at remote location( remote router port forward is set and working) . once the initial connection is made, the problem is the encoding server by default is looking for a local address of the Encoding Client - see attachment. The fix would be to edit the server with new distinct address & port back to Encoding client location. Or another option is address translation on the remote location Mikrotik Router ( I have yet to try)
Is there another way to get encoding server pointed to another address/port?

https://i.imgur.com/yZAGV4Z.png

[byteshare]

see attachment
Please try linking the image from an image hosting site such as imgur.com. Attaching photos is not the best method here.

[Atak_Snajpera]

I must be missing something... The easy part is getting Encoding Client to connect with the remote server (I simply add static address and the port to Distributed Encoding server list) - remote encoding server reports "Client established connection with server" so all is good connecting to server at remote location( remote router port forward is set and working) . once the initial connection is made, the problem is the encoding server by default is looking for a local address of the Encoding Client - see attachment. The fix would be to edit the server with new distinct address & port back to Encoding client location. Or another option is address translation on the remote location Mikrotik Router ( I have yet to try)
Is there another way to get encoding server pointed to another address/port?

https://i.imgur.com/yZAGV4Z.png

Btw. Do you have Access to shared folder from that machine?

[userx]

@Atak_Snajpera

Have you any recomendation on this (https://forum.doom9.org/showthread.php?p=1881341#post1881341)?

[Atak_Snajpera]

@Atak_Snajpera

Have you any recomendation on this (https://forum.doom9.org/showthread.php?p=1881341#post1881341)?

1) Outdated drivers
2) No enough VRAM (1GiB)

[slalom]

I have that issue from time to time where I need to strip the file of everything but the video or it doesn't add to RipBot.
Me too, but just loosing the audio will sufice

[rayers1060]

Btw. Do you have Access to shared folder from that machine?

I just realized shared folder is not accessible after my initial post. I was able to get address translation from remote location to work correctly. Currently both Encoding server and encoding client can correctly communicate commands between sites. The issue is the shared folder., The Encoding server cannot get chunks - see screenshot. I have other Encoding servers on the Local LAN and all is working well. The remote site cannot connect to the shared folder where the encoding client resides - Any ideas on making shared folder with transparent access??


https://imgur.com/mh6Cc1K

[Atak_Snajpera]

That's why people say to use VPN. You literally have to be a part of that LAN.
https://forums.tomshardware.com/threads/accessing-local-lan-via-internet.3389046/

[userx]

1) outdated drivers
2) no enough vram (1gib)

VRAM: 2GB,
Driver:07/2019

As mentioned the second pass is working as desired.

https://i.imgur.com/UhR1G1M.png

[ReinerSchweinlin]

How can I keep all audio when doing batches? Only one audio stream is copied/encoded.

[Viper714]

Okay Atak, All the files are uploaded in the same location. Please let me know if you need additional files....
Thanks again!

Atak,
Sorry for the late reply. Been away on business and just got back this past Friday.
I noticed that 1.25 was released. I tried that one and the first thing I noticed on one of my machines, it reported that Haali Splitter was not installed. Strange as it was always installed. Either way i installed it via the link RipBot provided.
I encoded:

The Chronicles of Riddick - Successful
The Chronicles of Narnia Prince Caspian - Successful
Chain Reaction - Successful

So far those three movies look great and are in sync. I will continue with the other movies and let you know. So far it appears promising and you fixed the issue. Curious what you may have changed?
Either way, thank you for your hard work and this great program!!!!

[kempodragon]

After Ripbot updated itself to the latest version, I did a test on several known files, and got this error: https://imgur.com/a/eKnHn1L . It only shows up on the server screen, not the log, which only says it could not open the file. The previous version encoded them perfectly, so I can only assume this is some sort of bug. I tried to download the newest version and do a fresh install, but the link on the page still only points to version 1.25.

[byteshare]

How can I keep all audio when doing batches? Only one audio stream is copied/encoded.
In batch mode you've tried setting it to the profile "x.x C O P Y S T R E A M"?
I usually run a batch afterwards that muxes in subs from the source to keep the MKV titles, and the encoded audio from HandBrake since an old version handles audio the way I like, so I might not be the most help.

After Ripbot updated itself to the latest version, I did a test on several known files, and got this error: https://imgur.com/a/eKnHn1L . It only shows up on the server screen, not the log, which only says it could not open the file. The previous version encoded them perfectly, so I can only assume this is some sort of bug. I tried to download the newest version and do a fresh install, but the link on the page still only points to version 1.25.
Try removing the job and recreating it since the update.
Also, try cropping the image :D Or WinKey + Shift + S to do a snip and select only the area you want for a screen grab.

[Viper714]

I am curious as to what these red indicators are.

https://i.imgur.com/RLzjto9.jpg

I ask because I am seeing some strange actions during encoding. Some chunks are stalling before even starting. It seems to eventually start but it takes about a minute or more. I have six servers active (3 different machines) and four servers are stuck on "starting..." Does not always happen but it is intermittent.
Any ideas??

[LigH]

No clue, just guessing ... depending on the source filter: An indexing phase?

[Atak_Snajpera]

Red = normal priority
Green = below normal
Blue = low priority

[kempodragon]

Byteshare, I already tried removing the job several times with no luck. Imgur.com didn't complain about the file dimensions or size so I decided to be lazy and just post whole screen. Similar problems have happened occasionally after an update so I find that grabbing the latest version and doing a clean install usually solves the problem. The link still only points to 1.24 so I'll have to wait until Atak updates the link. What I find a little strange is that the error message only appears on the server screen, it doesn't get recorded in the log.

[Atak_Snajpera]

I ask because I am seeing some strange actions during encoding. Some chunks are stalling before even starting. It seems to eventually start but it takes about a minute or more. I have six servers active (3 different machines) and four servers are stuck on "starting..." Does not always happen but it is intermittent.
Any ideas??

Take a look here https://forum.doom9.org/showthread.php?p=1881800#post1881800

[Viper714]

Take a look here https://forum.doom9.org/showthread.php?p=1881800#post1881800

Thanks Atak. Wow, quite a bit involved to encoding. Hopefully the compression method you recommend will be implemented.
:thanks:

[byteshare]

Byteshare, I already tried removing the job several times with no luck. Imgur.com didn't complain about the file dimensions or size so I decided to be lazy and just post whole screen. Similar problems have happened occasionally after an update so I find that grabbing the latest version and doing a clean install usually solves the problem. The link still only points to 1.24 so I'll have to wait until Atak updates the link. What I find a little strange is that the error message only appears on the server screen, it doesn't get recorded in the log.
Just open RipBot let it load the updated files, then close and reopen and it will install the updates.
-Do you get that index error with the same file on an older version of RipBot?
-It looks like there is either an issue accessing that file (can you confirm it is there?) or something might be up with the video.
-Try remuxing the file with only the video part and loading that.

[duffbeer]

Encoding with MDegrain2 seems to be broken since the last major update. The first few minutes are OK but then it becomes a slideshow of about 2 frames per second. I've tried a few different sources (AVC, VC-1, MPG2) but they all do the same thing.
Seems to be since the change to LSmash.

[Atak_Snajpera]

Encoding with MDegrain2 seems to be broken since the last major update. The first few minutes are OK but then it becomes a slideshow of about 2 frames per second. I've tried a few different sources (AVC, VC-1, MPG2) but they all do the same thing.
Seems to be since the change to LSmash.

That's funny because VC-1 is not being decoded by LSmash but by FFMS2 like in previous version. MDegrain2 has also not been updated.

[userx]

@Atak_Snajpera

Additional to this (https://forum.doom9.org/showthread.php?p=1881341#post1881341) issue I maybe have found the cause.
When I start a Job on the remote machine (local job), I run into the same problem if i switch 'x264 OpenCL acceleration' to Capeverde but runs without problems if i switch to None.
On a remote-job Encoding Server doesn't consider the None-setting in the 'RipBot264.ini' and want to use OpenCL accelleration.


This issue only occures in 1st-pass. 2nd-pass seems to run with OpenCL-Capeverde.
Is there an other possibility to tell Encoding Server not to use it?

[Atak_Snajpera]

Yes. Just run Encoding server with wrong device id
Example
EncodingServer.exe /x264-opencl-device-id 9

or you can just refresh job by clicking EDIT and Done

[duffbeer]

That's funny because VC-1 is not being decoded by LSmash but by FFMS2 like in previous version. MDegrain2 has also not been updated.

Sorry - my mistake. My test sources were AVC and MPG2 but I didn't try VC-1 yet.
I can send samples if it would help.

[Atak_Snajpera]

Why not?

[duffbeer]

Why not?

Not had time yet. I tried the same sources encoded with v1.24.1 and they were fine. I can test VC-1 if you want me to?

[userx]

/x264-opencl-device-id 9

Hero of the day!!
No idea why OpenCL isn't working but that solves my problem.
:thanks:

[byteshare]

Encoding with MDegrain2 seems to be broken since the last major update. The first few minutes are OK but then it becomes a slideshow of about 2 frames per second. I've tried a few different sources (AVC, VC-1, MPG2) but they all do the same thing.
Seems to be since the change to LSmash.
I've had similar results with AVC/HEVC inputs to 10bit x265 using AVISynth filters such as QTGMC and Megrain2. Without AVIsynth filters speeds seem close enough to the same that I can't tell, so I don' think it could be related to the updated x265.
Using the most recent update (including LSmash update), it seems to slow down the decoding process because extracting a frame is taking way longer when adding a job and using a lot of CPU while doing it.
--Maybe skip that step until someone clicks Edit > AviSynth? Or had Detect boarders on for cropping?
Unless something changed with FFMPEG, but I don't see an update to that since April of this year.
Another thing is I've noticed with filters my second Encoding Server on the same machine is taking way longer to start encoding than the first one, but with only one Encoding Server I'm not getting 100% CPU.
I'm not sure what I could send to help figure this out.
I tried a test with 1.24 vs fully updated 1.25 RipBot.
Same x265 version (one from the newest version of RipBot), same x265 profile, same filters for the same AVIsynth custom script.
For time used just one Encoding Server to encode the same chunk.
Custom: video=QTGMC(video,Preset="slower",FPSDivisor=2,NoiseProcess=0).ConvertBits(16).ConvertToStacked()
.SMDegrain(tr=3,thSAD=400,thSADC=150,refinemotion=false,contrasharp=true,plane=4,pel=2,
prefilter=4,Truemotion=False,chroma=true,hpad=32,vpad=32,str=2,amp=1,lsb_in=true,lsb=true,lsb_out=true)
.ConvertFromStacked().ConvertBits(8).FineDehalo().FastLineDarkenMOD4().LSFmod()(Added line breaks to keep the text from spanning too much on smaller screens.
Results:
confirmed extracting a frame was faster in 1.24
but for FPS of encoding:
1.25 0.20FPS
1.24 0.43FPS
...but, 1.24 was actually much faster than just FPS because the FPS isn't reporting correctly in 1.25 since the encoder isn't actually processing frames the whole time. It looks like the decoder stage is hanging and not passing frames to the encoder as quickly as in 1.24
--Sidenote, in 1.24 it is correctly splitting up the chunks for this file into 58 parts rather than 29 parts (but twice the number of frames roughly) and giving me the option for deinterlace drop down in the AVISynth settings.

Could the decoding slowness have something to do with the changes in the write buffer?

Another update:
So, it seems the extent of this slow down is a lot to do with the filter combintion I'm using as when I try without either SMDegrain or QTGMC the remaining filters are fine, or even when I try different settings such as prefilter 3 for SMD. If I swap SMDegrain for MD2 or Adaptive KNLmeansCL I also don't see the issue.
If I'm getting slow down without other heavy filters/settings it isn't enough to easily detect, but with 1.25 the issue with the above filters is exacerbated for one reason or another.

[duffbeer]

I've had similar results with AVC/HEVC inputs to 10bit x265 using AVISynth filters such as QTGMC and Megrain2. Without AVIsynth filters speeds seem close enough to the same that I can't tell, so I don' think it could be related to the updated x265.
Using the most recent update (including LSmash update), it seems to slow down the decoding process because extracting a frame is taking way longer when adding a job and using a lot of CPU while doing it.
--Maybe skip that step until someone clicks Edit > AviSynth? Or had Detect boarders on for cropping?
Unless something changed with FFMPEG, but I don't see an update to that since April of this year.
Another thing is I've noticed with filters my second Encoding Server on the same machine is taking way longer to start encoding than the first one, but with only one Encoding Server I'm not getting 100% CPU.
I'm not sure what I could send to help figure this out.
I tried a test with 1.24 vs fully updated 1.25 RipBot.
Same x265 version (one from the newest version of RipBot), same x265 profile, same filters for the same AVIsynth custom script.
For time used just one Encoding Server to encode the same chunk.
Custom: video=QTGMC(video,Preset="slower",FPSDivisor=2,NoiseProcess=0).ConvertBits(16).ConvertToStacked()
.SMDegrain(tr=3,thSAD=400,thSADC=150,refinemotion=false,contrasharp=true,plane=4,pel=2,
prefilter=4,Truemotion=False,chroma=true,hpad=32,vpad=32,str=2,amp=1,lsb_in=true,lsb=true,lsb_out=true)
.ConvertFromStacked().ConvertBits(8).FineDehalo().FastLineDarkenMOD4().LSFmod()(Added line breaks to keep the text from spanning too much on smaller screens.
Results:
confirmed extracting a frame was faster in 1.24
but for FPS of encoding:
1.25 0.20FPS
1.24 0.43FPS
...but, 1.24 was actually much faster than just FPS because the FPS isn't reporting correctly in 1.25 since the encoder isn't actually processing frames the whole time. It looks like the decoder stage is hanging and not passing frames to the encoder as quickly as in 1.24
--Sidenote, in 1.24 it is correctly splitting up the chunks for this file into 58 parts rather than 29 parts (but twice the number of frames roughly) and giving me the option for deinterlace drop down in the AVISynth settings.

Could the decoding slowness have something to do with the changes in the write buffer?

I think you may have misunderstood my problem. The issue I was referring to are with the encoded output file.
Encoding speed seems a little slower but the biggest problem is when I play back a file that was encoded with 1.25 using MDegrain2. The first 10-20 mins play back OK but then it plays at roughly 2fps. The audio is not affected and continues to play correctly.
I have tried the same source files on 1.24.1 and there is no problem so I believe it has something to do with LSmash.

The other big problem since 1.25 is that it takes roughly 2 minutes to auto crop. Generating a new preview frame also takes at least 1 minute. If I do exactly the same thing with 1.24.1 it crops in about 3-4 seconds.

It would be useful to know if anyone else has these issues. I've never had any serious issues with auto updates in the past, but this is the first time I'm going to have to ignore the latest version as it's virtually unusable in it's current form.
1.24.1 still works perfectly for me.

I've been using RipBot since 2012 and I think it's fantastic but the latest update seems totally broken to me.

[ReinerSchweinlin]

In batch mode you've tried setting it to the profile "x.x C O P Y S T R E A M"?

Thanx for your answer.
Yes, of course I did that :) And it only keeps the first audio stream in batch mode. It has always been like this since I started using RipBot. It would be lovely if RipBot could simply keep all streams when doing batches :)

[ReinerSchweinlin]

Thanx for your answer.
Yes, of course I did that :) And it only keeps the first audio stream in batch mode. It has always been like this since I started using RipBot. It would be lovely if RipBot could simply keep all streams when doing batches :)
I have to add: Often I don´t want to copy the streams, but encode them to something different (1. stream englisch AC3, 2. Stream japanese DTS -> both to AAC HE)... Either way, in batch mode, only the first stream is transcoded....

[Ronski]

Hi, I have a bit of a weird problem. I've had this happen in the past and I've simply restarted Windows and its been fine, now it does it all the time, my PC did update to v1903 of Windows 10 (64bit) tonight, so not sure if that's any relevance, I've also turned off my antivirus just in case it was that, but made no difference.

I have two encoding servers set to start with Ripbot, neither are visible in the system tray, neither can Ripbot connect to them although it does connect to the instance on my other PC, but if I open task manager they are both there in the process list, if I open another instance of EncodingServer.exe then that also appears in the process list but still not in the tray or on the desktop.

Any thoughts please?

https://live.staticflickr.com/65535/48538532156_b5157579c6_o.png (https://flic.kr/p/2gXbEy5)


Edit: Actually I think this may be something to do with Avast, I copied the encodingserver.exe to another directory and ran it, first windows firewall asked if I wanted to permit access, after permitting the encoding server opened, but then it had Avast CyberCapture above it, then promptly disappeared - I'll do some more testing. Odd thing is Avast is running on the other PC as well.

[byteshare]

I have to add: Often I don´t want to copy the streams, but encode them to something different (1. stream englisch AC3, 2. Stream japanese DTS -> both to AAC HE)... Either way, in batch mode, only the first stream is transcoded....
Mostly to keep the mkv title of subs I process subs and audio outside of RipBot and mux it all a back at the end with a batch file. If you want any help with that let me know.

[duffbeer]

Anyone else experiencing cropping speed problems and MDegrain2 problems with v1.25?

I tested a VC-1 source and it was perfect but AVC or MPG2 cannot be encoded with MDegrain2 since the last update.

I'm happy to accept there may be a problem with my specific installation but 1.24 works perfectly.
Is there anything else I should do after the auto update has applied 1.25?

[george84]

Very simple avs file

x = ImageSource("testchart1.jpg", 0, 0, fps=24, use_DevIL = true, info=false, pixel_type = "RGB48")
x=ConvertToPlanarRGB(x)
x = z_ConvertFormat(x,resample_filter="bicubic", pixel_type="RGBPS", colorspace_op="rgb:srgb:709:f=>rgb:709:709:f")
x=ConvertToYV12(x)
x = Loop(x,120)
x

is added to RipBot264. It goes to Please Wait ... Gathering Information...

and never comes back. Windows10, RipBot 1.25.0 with all updates done. AVSMeter processes file without problems.
Problem is not important for me. Just FYI.

[Viper714]

Hi, I have a bit of a weird problem. I've had this happen in the past and I've simply restarted Windows and its been fine, now it does it all the time, my PC did update to v1903 of Windows 10 (64bit) tonight, so not sure if that's any relevance, I've also turned off my antivirus just in case it was that, but made no difference.

I have two encoding servers set to start with Ripbot, neither are visible in the system tray, neither can Ripbot connect to them although it does connect to the instance on my other PC, but if I open task manager they are both there in the process list, if I open another instance of EncodingServer.exe then that also appears in the process list but still not in the tray or on the desktop.

Any thoughts please?

https://live.staticflickr.com/65535/48538532156_b5157579c6_o.png (https://flic.kr/p/2gXbEy5)


Edit: Actually I think this may be something to do with Avast, I copied the encodingserver.exe to another directory and ran it, first windows firewall asked if I wanted to permit access, after permitting the encoding server opened, but then it had Avast CyberCapture above it, then promptly disappeared - I'll do some more testing. Odd thing is Avast is running on the other PC as well.

I noticed you are running Logitech Software. See my post here (https://forum.doom9.org/showpost.php?p=1879997&postcount=17029). Hope this helps..

There is a post after mine from byteshare (Here (https://forum.doom9.org/showpost.php?p=1880045&postcount=17034)) to update to a different version of the Logitech Software as well. I haven't done that yet but plan on doing it soon. Just need to make sure that I do not loss my profiles as I have a few of them for my keyboard...

[Ronski]

Viper714, thank you so much for the reply, disabled the Logitech framework and Ripbot now works. I only have the Logitech software installed for my MX518 mouse (not compatible with ghub), and that seems to work fine without the frame work running. :thanks:

[howzz]

i got a bit of an odd question for the author, i noticed that ever since v 1.25, with Lmash decoding i assume, the indexing process takes twice as long than before. i built my encoding machine purposely with all Nvme drives to speed up the demux and index process, but it seems with the latest Lmash update, after the demux, and after index (which btw you can tell that sequential disk read/write are both at max), "gathering information/Extracting Frames" seems to be doing some additional CPU crunching without much disk usage or if at all and that "gathering information/extracting frames" process now takes wayyyyyy longer than before.

i confirmed the performance difference by comparing to v. 1.23.1, which was my fall-back version.

with v. 1.23.1, after demux and index, gathering information only takes about 5 seconds on my Nvme doing HDR H265 source, now with v. 1.25 when it gets to the Gathering Information and extracting frames, it takes like 5~10 or so mins? i checked the task manager while it's doing its thing to find out why it's taking this longer, and looks like it's launching the ffprobe AGAIN to do something. FFProbe was already launched once when demuxing was taking place but why now again?

thoughts?

[Atak_Snajpera]

What is the size of .lwi file in job folder?

[howzz]

What is the size of .lwi file in job folder?

about 1.25GB

is that good or bad

[howzz]

i tried v 1.23.1 again, and it doesn't produce .lwi file at all. so i guess v 1.25 is producing this .lwi file that's taking forever. what exactly does it do, and why do we need it? this makes me want to just go back to v 1.23.1 but v. 1.25 has the latest version of X265 (release v30 i beleive) with lots of improvements in the codec.

[stax76]

i tried v 1.23.1 again, and it doesn't produce .lwi file at all. so i guess v 1.25 is producing this .lwi file that's taking forever. what exactly does it do, and why do we need it? this makes me want to just go back to v 1.23.1 but v. 1.25 has the latest version of X265 (release v30 i beleive) with lots of improvements in the codec.

Source filters like ffms2, l-smash and dgdecnv create a index file before they open a source file, it's used to enable seeking. l-smash has a filter that can open mp4 without creating an index file, sadly not working with mkv.

[howzz]

Source filters like ffms2, l-smash and dgdecnv create a index file before they open a source file, it's used to enable seeking. l-smash has a filter that can open mp4 without creating an index file, sadly not working with mkv.

so does this mean that you can use a player to play the movie and able to do fast seek? if so that's an awesome added feature that i am willing to take the performance hit (waiting for it to create index).

but maybe i misunderstood you.

[stax76]

so does this mean that you can use a player to play the movie and able to do fast seek? if so that's an awesome added feature that i am willing to take the performance hit (waiting for it to create index).

but maybe i misunderstood you.

It's unrelated to playback, source filters create an index helping them to ensure frame accuracy for random frame access that some filters might do. There are people who understand and can explain this better than me.

[howzz]

Update:

Strangely. there's a new update that just rolled out a few mins ago. i launched ripbot again and it udpated itself. so i looked into the update log, and looks like either Atak may have enabled a new update for Lmash pushed a new version update to ripbot. either way, Lmash got updated, and now when it gets to Gathering Information/Extracting frames, it does so much faster. i also noticed that it compressed the index with the new update. so instead of the massive 1.25GB index file, .lwi is now 30MB small. and you can tell that when it compressed it because just right after indexing and before Gathering information, you see the ripbot status bar briefly shows Compressing Index. it then quickly proceeds to Gathering information and Extracting Frame and only take 3 or so mins now.

so whatever Atak did. thanks!. or whatever Lmash did, thanks!

==========
2019-08-17 13:38:25 : Update for [lsmash] detected
2019-08-17 13:38:25 : Downloading file http://atak-snajpera.5v.pl/ripbot264update/lsmash.zip to D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\lsmash.zip
2019-08-17 13:38:26 : [SUCCESS] File D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\lsmash.zip saved!
2019-08-17 13:38:26 : CRC32 value has not changed for [7z]. Update is not required.
2019-08-17 13:38:26 : Downloading finished.
2019-08-17 16:37:34 : =========================[UPDATER ACTIVATED]=========================
2019-08-17 16:37:34 : Installing updates...
2019-08-17 16:37:39 : [SUCCES] D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\core.zip has been correctly extracted!
2019-08-17 16:37:48 : [SUCCES] D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\ffmpeg.zip has been correctly extracted!
2019-08-17 16:37:49 : [SUCCES] D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\lsmash.zip has been correctly extracted!
2019-08-17 16:37:49 : Installation complete.
2019-08-17 16:37:55 : Next check after 2019-08-18 13:38:18
2019-08-17 16:38:42 : Next check after 2019-08-18 13:38:18

[howzz]

It's unrelated to playback, source filters create an index helping them to ensure frame accuracy for random frame access that some filters might do. There are people who understand and can explain this better than me.

thanks, your explanation actually makes sense to me. :)

[FuzzyNutz]

As of the last few days, playback of some of my blu-ray source videos encoded with RB have occasional flickering, where video frames seem to pause or jump back and forward within fractions of seconds. After the incidents, playback returns to normal and the audio is still synced correct.

[ReinerSchweinlin]

Mostly to keep the mkv title of subs I process subs and audio outside of RipBot and mux it all a back at the end with a batch file. If you want any help with that let me know.
Hey.
Thank you for the offer. I had a closer look into the temp folders... The two audio streams are extracted, audiolist.txt lists two streams and audiostreams.cmd lists to prozess 3 streams.
What´s missing is content in "encodeaudio2.cmd" - it has zero bytes.

[brumsky]

I'm having an odd issue with ripbot. It's taking 5-15 minutes to start an encode chunk every time. I already did the update which compresses the lwi file down to around 30MBs.

Something I've noticed is that when the encoding client is starting a new chunk. The system process pegs my network connection - currently 1Gbps. While the disk usage is very low. So it'll start disk usage will hit 20-30MBps then drops to 1-2MB/s while the network is pegged at 800-900 Mb/s. it'll stay like that for quite some time. This is going from a R7 1700 to a AMD 3900x via Gigabit ethernet at the moment.

I've also confirmed the system running the encoding server matches the system process behavior.

I've also tried this on another machine same issue - 16c Xeon v4.

I understand that the lwi file is larger in general which takes it longer to copy. This seems abnormally long given the lwi file uncompressed is usually around 1.3GBs for a 4K movie.

Thoughts?

[Pino72]

Same for me, the last days were a nightmare....DE that always worked since months suddenly broken, servers not starting at all, big time delays when rendering chunks like you described, a looooong wait for Autocrop and so on.

I hope these issues get fixed soon since I really think ripbot is the best tool to convert uhd 4K hdr files.

At least since today I am able to use DE again on my two machines and even after aborting he picks up where it left....which also seemed broken a day or two ago. Really strange.

[brumsky]

Digging into this further, I can see that it is reading the video.mkv file. It appears to be transferring the whole file. that is the only thing that makes sense given how small the new lwi files is after compressing.

[slalom]

Same here, veeeery long delays, the network at 97% (gigabit)

[BLKMGK]

If you are going to use KNLMeansCL in DE mode then you must:
1) have decent gpus in all PC's. GeForce 210 is just a crap GPU (Supports up to OpenCL 1.1 [knlmeansCL requires 1.2] and only two compute units!).
2) manually set device id in encoding server if you have more than one GPU. One of your machines has Intel GPU and GeForce 1070 for example.
This will tell encodingserver to use GeForce 1070 (faster)
EncodingServer.exe /knlmeanscl-opencl-device-id 1

In your case It might be a good idea to run one server using Intel GPU and other GeForce 1070 .
EncodingServer.exe /knlmeanscl-opencl-device-id 0
EncodingServer.exe /knlmeanscl-opencl-device-id 1

This way you will be using literally everything what can compute ;) CPU+iGPU+dGPU

If you want you can also try running KNLMeansCL on CPU but be warned it will be veeeeeeeerrrrrrryyyyy slooooooooow.
EncodingServer.exe /knlmeanscl-opencl-device-type CPU

Okay, I've now installed an OpenCL1.2 compatible 1050 Ti card. I can encode on THAT machine with KNLMeansCL just fine but when I attempt to use that as a client with an existing machine that can already handle KNLMeansCL the Encoding Server accepts the job but never actually processes it :( Here's the OpenCL settings page, note that for jobs not using this filter this client works just fine. Am I missing something? This client was updated as of this afternoon so it's current. This should work right?

P.S. I tried piping the filter to the CPU, it didn't work either sadly, I'd have loved to see how the 3900X handled that :D

https://i.postimg.cc/ZqJVKM0B/openCL.jpg

Edit: I may have found something - it looks like the old index files and the new index files may not be playing well together? My GTX 680 card claims OpenCL 1.2 but after a lengthy period of time creating the index it fails to "open file" like my previous attempts and then tries to recreate the index AGAIN :-O I will try this on the machine with the 1050 card as soon as I'm able to gain access to it again.

[BLKMGK]

Whoa! Just threw a brand new unfiltered job into the queue. After EACH block it's rebuilding an index file! It's literally taking longer to build the index file than it is to compress the block - d'oh! This is v1.25 using Encoding Server v1.15.2.0 just updated tonight.

[Pino72]

Funnily, chunks in DE on the remote machine start more or less immediately on the local machine it takes 1-2 minutes....is this supposed to be normal? Did not have the rebuilding of the index luckily.

[ReinerSchweinlin]

I recognized something:

- The new index files are put into the source directory instead of the TEMP dir.
- after encoding and deleting all the jobs, the files still are recognized as "used" by windows.
- on some machines in the local network, the creating of the LWI Files after "Start" takes several minutes (the lokal machine and two others - all others are finished after a few seconds)

[duffbeer]

Has anyone else tried MDegrain2 since the update to LSmash? On my PC, the encoded output is OK for the first few minutes and then it becomes a slideshow of about 2fps.

[BLKMGK]

I recognized something:

- The new index files are put into the source directory instead of the TEMP dir.
- after encoding and deleting all the jobs, the files still are recognized as "used" by windows.
- on some machines in the local network, the creating of the LWI Files after "Start" takes several minutes (the lokal machine and two others - all others are finished after a few seconds)

In my case the LWI files used to be in the source directory but now I can no longer find where they're being stored on disk, perhaps this is why they rebuild with each chunk? Jobs that should complete in under an hour easily are taking quite a bit longer. A queue that should've completed overnight is a fraction of the way complete - d'oh! Even looking at task manager and disk stats I cannot see where the LWI file is being created...

https://i.postimg.cc/zXdJXT37/image.png

[ReinerSchweinlin]

I just looked and can´t find it either.. Strange...

edit: ahhhhhh :

appdata/local/temp/encodingserver1



... there i found some on one of the machines (which takes up a few seconds for the index files.)

I´ve noticed, that the "master machine" (which is one of the ones in the swarm needing about 5min to build the index files compared to around 20 sec) had about 50% to 80% GPU load on the csrss.exe while the index lwmi was building...

Just checked another machine (small 4 Core Celeron with 64bit win10), which does exactly the same - high GPU load on csrss while the index was built... This machine also takes minutes on the same 20min file I am encoding to build the index file.

Checked third machine: Dual Xeon 5660 - no GPU... this one was done in 5 seconds...


another thing: The video file always is copied completely into the TEMP directory - renamed to video.mkv. (The setting is "off" in the settings). Maybe this is on purpose and good for jobs where the source is stored on a usb-hd or slow SMD share - but doing a lot of jobs from a local harddrive I´d guess it would be enough to not copy the whole video - it fills up the temp drive a lot (and after all, we have all these nice indexes now...)

[ReinerSchweinlin]

Update:

Tested another PC - old Core2Duo - no GPU: This one also takes a long time to build the index - not much difference to the master machine (i5 kaby Lake)... So "having a GPU" can´t be the issue :)

It seems, the dual-xeon is so much faster since it has A LOT of cache build in, so it can index much faster - just guessing here.

[Atak_Snajpera]

Some tips from me.
1) Make sure you have been fully updated and ALL your remote PC have latest version of EncodingServer.exe running
https://i.imgsafe.org/ac/ac34cb063c.png
2) Start with new fresh job

Compressed index file is stored in JOB folder as video.mkv.lwi.7z. This file is next automatically decompressed on fly to %TEMP%/encodingserverX by EncodingServer.exe. Then EncodingServer modifies cachefile="" in X.avs file with proper path to extracted index file.
I did this to save few seconds on my slow LAN100/Wi-FI N network (real transfer is only ~10MiB/s). If Index file is 30MiB then I save 3s from starting procedure. In order to save few extra seconds I also compressed ffmpeg.exe with UPX (20MiB vs 60MiB).

[byteshare]

Hey.
Thank you for the offer. I had a closer look into the temp folders... The two audio streams are extracted, audiolist.txt lists two streams and audiostreams.cmd lists to prozess 3 streams.
What´s missing is content in "encodeaudio2.cmd" - it has zero bytes.
I think you misunderstood me. I deal with the subs and audio outside of RipBot (another application) and then mux it all with a back to the encoded file from RipBot.

[byteshare]

I think you may have misunderstood my problem. The issue I was referring to are with the encoded output file.
Encoding speed seems a little slower but the biggest problem is when I play back a file that was encoded with 1.25 using MDegrain2. The first 10-20 mins play back OK but then it plays at roughly 2fps. The audio is not affected and continues to play correctly.
I have tried the same source files on 1.24.1 and there is no problem so I believe it has something to do with LSmash.

The other big problem since 1.25 is that it takes roughly 2 minutes to auto crop. Generating a new preview frame also takes at least 1 minute. If I do exactly the same thing with 1.24.1 it crops in about 3-4 seconds.

It would be useful to know if anyone else has these issues. I've never had any serious issues with auto updates in the past, but this is the first time I'm going to have to ignore the latest version as it's virtually unusable in it's current form.
1.24.1 still works perfectly for me.

I've been using RipBot since 2012 and I think it's fantastic but the latest update seems totally broken to me.

Anyone else experiencing cropping speed problems and MDegrain2 problems with v1.25?

I tested a VC-1 source and it was perfect but AVC or MPG2 cannot be encoded with MDegrain2 since the last update.

I'm happy to accept there may be a problem with my specific installation but 1.24 works perfectly.
Is there anything else I should do after the auto update has applied 1.25?

Very simple avs file

x = ImageSource("testchart1.jpg", 0, 0, fps=24, use_DevIL = true, info=false, pixel_type = "RGB48")
x=ConvertToPlanarRGB(x)
x = z_ConvertFormat(x,resample_filter="bicubic", pixel_type="RGBPS", colorspace_op="rgb:srgb:709:f=>rgb:709:709:f")
x=ConvertToYV12(x)
x = Loop(x,120)
x

is added to RipBot264. It goes to

and never comes back. Windows10, RipBot 1.25.0 with all updates done. AVSMeter processes file without problems.
Problem is not important for me. Just FYI.


Update:

Strangely. there's a new update that just rolled out a few mins ago. i launched ripbot again and it udpated itself. so i looked into the update log, and looks like either Atak may have enabled a new update for Lmash pushed a new version update to ripbot. either way, Lmash got updated, and now when it gets to Gathering Information/Extracting frames, it does so much faster. i also noticed that it compressed the index with the new update. so instead of the massive 1.25GB index file, .lwi is now 30MB small. and you can tell that when it compressed it because just right after indexing and before Gathering information, you see the ripbot status bar briefly shows Compressing Index. it then quickly proceeds to Gathering information and Extracting Frame and only take 3 or so mins now.

so whatever Atak did. thanks!. or whatever Lmash did, thanks!

==========
2019-08-17 13:38:25 : Update for [lsmash] detected
2019-08-17 13:38:25 : Downloading file http://atak-snajpera.5v.pl/ripbot264update/lsmash.zip to D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\lsmash.zip
2019-08-17 13:38:26 : [SUCCESS] File D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\lsmash.zip saved!
2019-08-17 13:38:26 : CRC32 value has not changed for [7z]. Update is not required.
2019-08-17 13:38:26 : Downloading finished.
2019-08-17 16:37:34 : =========================[UPDATER ACTIVATED]=========================
2019-08-17 16:37:34 : Installing updates...
2019-08-17 16:37:39 : [SUCCES] D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\core.zip has been correctly extracted!
2019-08-17 16:37:48 : [SUCCES] D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\ffmpeg.zip has been correctly extracted!
2019-08-17 16:37:49 : [SUCCES] D:\DVDRip Work Files\Programs\RipBot264v1.24.0\Updates\lsmash.zip has been correctly extracted!
2019-08-17 16:37:49 : Installation complete.
2019-08-17 16:37:55 : Next check after 2019-08-18 13:38:18
2019-08-17 16:38:42 : Next check after 2019-08-18 13:38:18

As of the last few days, playback of some of my blu-ray source videos encoded with RB have occasional flickering, where video frames seem to pause or jump back and forward within fractions of seconds. After the incidents, playback returns to normal and the audio is still synced correct.


I'm having an odd issue with ripbot. It's taking 5-15 minutes to start an encode chunk every time. I already did the update which compresses the lwi file down to around 30MBs.

Something I've noticed is that when the encoding client is starting a new chunk. The system process pegs my network connection - currently 1Gbps. While the disk usage is very low. So it'll start disk usage will hit 20-30MBps then drops to 1-2MB/s while the network is pegged at 800-900 Mb/s. it'll stay like that for quite some time. This is going from a R7 1700 to a AMD 3900x via Gigabit ethernet at the moment.

I've also confirmed the system running the encoding server matches the system process behavior.

I've also tried this on another machine same issue - 16c Xeon v4.

I understand that the lwi file is larger in general which takes it longer to copy. This seems abnormally long given the lwi file uncompressed is usually around 1.3GBs for a 4K movie.

Thoughts?

Same for me, the last days were a nightmare....DE that always worked since months suddenly broken, servers not starting at all, big time delays when rendering chunks like you described, a looooong wait for Autocrop and so on.

I hope these issues get fixed soon since I really think ripbot is the best tool to convert uhd 4K hdr files.

At least since today I am able to use DE again on my two machines and even after aborting he picks up where it left....which also seemed broken a day or two ago. Really strange.


I recognized something:

- The new index files are put into the source directory instead of the TEMP dir.
- after encoding and deleting all the jobs, the files still are recognized as "used" by windows.
- on some machines in the local network, the creating of the LWI Files after "Start" takes several minutes (the lokal machine and two others - all others are finished after a few seconds)


Has anyone else tried MDegrain2 since the update to LSmash? On my PC, the encoded output is OK for the first few minutes and then it becomes a slideshow of about 2fps.

Quoted all of that since it seems some have not updated RipBot since the 1.25 update. If your LSmash files are not being stored in the job folder and you're not seeing .7z files for LSmash you need to update.
-Close RipBot Open, wait for the update to download files, close, reopen and the update will be installed. You might need to do this more than once if you didn't wait long enough.

Once you're on the most recent update you should then see if this fixed any of your issues.
-Sidenote. It would be helpful to change the point releases with these updates such as 1.25.xx, ie 1.25.02 or something

I did some testing in between 1.25 and the most recent update:
https://forum.doom9.org/showthread.php?p=1881866#post1881866
I'll need to do more testing myself to see if:
-Any of the filter combination issues I was having got fixed
-Extracting a frame is faster
-Auto Crop is faster
-See any weird frames jumping around (but audio in sync when it stops) on some files when using QTGMC that I hadn't seen until LSmash.

[ReinerSchweinlin]

I think you misunderstood me. I deal with the subs and audio outside of RipBot (another application) and then mux it all with a back to the encoded file from RipBot.
Yes, that was perfectly clear :) Thanx for the offer! For now I will first deal with those files having only one language - maybe Ripbot will be able to handle it in the future - if not, I will gladly come back to your offer.

@Atak
Thanx. Of course, everything is up to date :) At least a few hours ago. Started ripbot on every machine three times, waited 5 minutes in between to fully catch all updates - then started to investigate..

I think I found the issue of some machines doing very slow "indexing" mentioned above:

Close the encoding server windows (which is displaying the "indexing" procedure from 0 to 100%) - this did the trick for me - all machines now only take some seconds - the slower ones having the lowest cpu/bandwith... Machines which took minutes before now are starting after a few seconds..

Nice :)

So it seems the screen-output of ther encoding server window is causing this?

[byteshare]

Thanx. Of course, everything is up to date :) At least a few hours ago. Started ripbot on every machine three times, waited 5 minutes in between to fully catch all updates - then started to investigate..
Core should be Version: 2019.08.17
With the above version:
-Extracting a frame is faster
-Auto Crop is faster

Still need to test:
-Any of the filter combination issues I was having got fixed
-See any weird frames jumping around (but audio in sync when it stops) on some files when using QTGMC that I hadn't seen until LSmash.

[ReinerSchweinlin]

@byteshare
Yes, we are on the same page, ah version :)

Did some more testing - while encoding some old anime, I noticed this:

source video (resized to fit width of the target for better comparison)

https://forum.doom9.org/attachment.php?attachmentid=16946&stc=1&d=1566241893
encoded video (q22, slow, no filter, resized to match 720p, HEVC, 10 Bit)
https://forum.doom9.org/attachment.php?attachmentid=16945&stc=1&d=1566241893

Its not exactly the same frame, but that doesn´t matter..

Notice the 1 pixel dots between the soft noise in the picture - and in the middle of the white "snake-like" thing...

These artefacts seem to come from resizing, I have seen them quite a lot when encoding animes..

[byteshare]

@byteshare
Yes, we are on the same page, ah version :)

Did some more testing - while encoding some old anime, I noticed this:

source video (resized to fit width of the target for better comparison)

https://forum.doom9.org/attachment.php?attachmentid=16946&stc=1&d=1566241893
encoded video (q22, slow, no filter, resized to match 720p, HEVC, 10 Bit)
https://forum.doom9.org/attachment.php?attachmentid=16945&stc=1&d=1566241893

Its not exactly the same frame, but that doesn´t matter..

Notice the 1 pixel dots between the soft noise in the picture - and in the middle of the white "snake-like" thing...

These artefacts seem to come from resizing, I have seen them quite a lot when encoding animes..

Link the images from an image hosting site rather than uploading directly to this forum, such as Imgur.com

That said what are you using to resize the video? Have you tried: nnedi3_resize16
Also, are you using other filters?

[slalom]

Some tips from me.
1) Make sure you have been fully updated and ALL your remote PC have latest version of EncodingServer.exe running
https://i.imgsafe.org/ac/ac34cb063c.png
2) Start with new fresh job

Compressed index file is stored in JOB folder as video.mkv.lwi.7z. This file is next automatically decompressed on fly to %TEMP%/encodingserverX by EncodingServer.exe. Then EncodingServer modifies cachefile="" in X.avs file with proper path to extracted index file.
I did this to save few seconds on my slow LAN100/Wi-FI N network (real transfer is only ~10MiB/s). If Index file is 30MiB then I save 3s from starting procedure. In order to save few extra seconds I also compressed ffmpeg.exe with UPX (20MiB vs 60MiB).
I downloaded and extracted the files from the first page
I ran that to update to the latest files, checked the files with yours
I removed all jobs from previous version
I copied the updated folder everywere and added 5 jobs

Same thing, massive network usage

[BLKMGK]

Some tips from me.
1) Make sure you have been fully updated and ALL your remote PC have latest version of EncodingServer.exe running
https://i.imgsafe.org/ac/ac34cb063c.png
2) Start with new fresh job
<snip>

started with a fresh job, all CRC are correct, i see indexes being built on each client after each block. i'm currently only running 2x clients with remote access. I'm not sure what to try, these files are small and arent using filters. In 8 hours i processed 4 30min long 720p video that would normally process in under 3mins each :scared:

ill try a fresh install when i can lay hands kn the boxes in person, happy to provide anything to troubleshoot though. Still would love to solve my GPU issue filtering too!

[brumsky]

I made sure all CRCs matched on both of my machines.

I am still getting high network usage and now the encoding server shows it is creating the index file. Here is print out of it - if it helps.

https://pastebin.com/WB2ACdfg

thoughts??

[FuzzyNutz]

Quoted all of that since it seems some have not updated RipBot since the 1.25 update. If your LSmash files are not being stored in the job folder and you're not seeing .7z files for LSmash you need to update.
-Close RipBot Open, wait for the update to download files, close, reopen and the update will be installed. You might need to do this more than once if you didn't wait long enough.

Once you're on the most recent update you should then see if this fixed any of your issues.
-Sidenote. It would be helpful to change the point releases with these updates such as 1.25.xx, ie 1.25.02 or something

I did some testing in between 1.25 and the most recent update:
https://forum.doom9.org/showthread.php?p=1881866#post1881866
I'll need to do more testing myself to see if:
-Any of the filter combination issues I was having got fixed
-Extracting a frame is faster
-Auto Crop is faster
-See any weird frames jumping around (but audio in sync when it stops) on some files when using QTGMC that I hadn't seen until LSmash.

RipBot264 updated today, yet it's still producing videos with flickering frames or "frames jumping around".

[BLKMGK]

I've erased everything, dropped back to v1.24, and turned off auto-updating for now :( It seems like others are experiencing the same issues with index building I was so I'll wait a bit, hopefully address filtering via GPU not working on clients (for me) later. I suspect the fix won't be super difficult to solve the indexing but I've got a stack of videos waiting for me and cannot proceed at such a slow pace...

[userx]

Hello
After last update I'm not able to start a DE job. On local and remote machine EncodingServer runs into
Encoding started...
""\\USERX-MASTER\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\USERX-MASTER\RipBot264temp\job1\Chunks\1.avs" -strict -1 -f yuv4mpegpipe - | "\\USERX-MASTER\Ripbot264temp\tools\x264\x264_x64.exe" --seek 0 --colorprim bt709 --transfer bt709 --colormatrix bt709 --opencl --opencl-device 0 --opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead_1.clbin" --pass 1 --bitrate 3000 --stats "\\USERX-MASTER\RipBot264temp\job1\Chunks\1.stats" --fps 24000/1001 --force-cfr --min-keyint 24 --keyint 240 --frames 1440 --sar 1:1 --level 4.1 --aud --nal-hrd vbr --vbv-bufsize 25000 --vbv-maxrate 25000 --b-pyramid none --ref 5 --stdin y4m --output NUL -"
y4m [error]: bad sequence header magic
x264 [error]: could not open input file


Has anybody an idea to fix it?

[byteshare]

RipBot264 updated today, yet it's still producing videos with flickering frames or "frames jumping around".
Are you using filters? I've seen the issue with QTGMC.
I'm not having that issue with AVC/HEVC source either without filters or with LSFmod or FastLineDarkenMod.
I need to test with more filters besides QTGMC.
I've erased everything, dropped back to v1.24, and turned off auto-updating for now :( It seems like others are experiencing the same issues with index building I was so I'll wait a bit, hopefully address filtering via GPU not working on clients (for me) later. I suspect the fix won't be super difficult to solve the indexing but I've got a stack of videos waiting for me and cannot proceed at such a slow pace...
I'm not having the issue...so maybe something more specific to the default startup commands for the encoding servers?
I'm using:
/port 1000 /minimize /priority low /restart-if-no-progress 8

[byteshare]

Hello
After last update I'm not able to start a DE job. On local and remote machine EncodingServer runs into
Encoding started...
""\\USERX-MASTER\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i
"\\USERX-MASTER\RipBot264temp\job1\Chunks\1.avs" -strict -1 -f yuv4mpegpipe - |
"\\USERX-MASTER\Ripbot264temp\tools\x264\x264_x64.exe" --seek 0 --colorprim bt709 --transfer bt709 --colormatrix bt709 --opencl --opencl-device 0 --opencl-clbin
"C:\Users\UserX\AppData\Local\Temp\x264_lookahead_1.clbin" --pass 1 --bitrate 3000
--stats "\\USERX-MASTER\RipBot264temp\job1\Chunks\1.stats" --fps 24000/1001
--force-cfr --min-keyint 24 --keyint 240 --frames 1440 --sar 1:1 --level 4.1 --aud
--nal-hrd vbr --vbv-bufsize 25000 --vbv-maxrate 25000 --b-pyramid none --ref 5
--stdin y4m --output NUL -"
y4m [error]: bad sequence header magic
x264 [error]: could not open input file


Has anybody an idea to fix it?
Are you able to browse to the share and see the file? Could be an issue with the share.

[ReinerSchweinlin]

Link the images from an image hosting site rather than uploading directly to this forum, such as Imgur.com

That said what are you using to resize the video? Have you tried: nnedi3_resize16
Also, are you using other filters?
thanx for your reply!

I´d rather leave them in the forum - external Image hosting tends to ditch pictures after a while - several older threads are almost unusable because people put the images on imageshack - now they are gone...

Or is there a particular reason why it´s not recommended to save them directly in the forum?

I used the default resize funktion in Ripbot. Looking at the Avisynthscript, it is spline resize - I am not at the pc right now, so I´d have to check exactly, but I think I remember it was spline36, using some plugin for multihtreading.... The default one.. didn´t change anything. no custom scripts.

No, no filters, just resizing.

[Atak_Snajpera]

thanx for your reply!

I´d rather leave them in the forum - external Image hosting tends to ditch pictures after a while - several older threads are almost unusable because people put the images on imageshack - now they are gone...

Or is there a particular reason why it´s not recommended to save them directly in the forum?

I used the default resize funktion in Ripbot. Looking at the Avisynthscript, it is spline resize - I am not at the pc right now, so I´d have to check exactly, but I think I remember it was spline36, using some plugin for multihtreading.... The default one.. didn´t change anything. no custom scripts.

No, no filters, just resizing.

Your post will be forgoten because approval may take weeks.

[BLKMGK]

<snip>

I'm not having the issue...so maybe something more specific to the default startup commands for the encoding servers?
I'm using:
/port 1000 /minimize /priority low /restart-if-no-progress 8

I don't modify the commandline for launch at all. I do use port 1000, low priority, and for restart I use the GUI buttons from the encoding client but generally leave restart alone except when I'm encoding something with KNLMeansCL since this only ever encodes blocks on my host machine and not clients. At least two clients have OpenCL1.2 capable cards installed and configured but reject blocks anyway. The videos encode on those machines standalone fine though :confused:

For now I'm back in business with the older client. When the current round of jobs is completed I'll backup my files and attempt again with a newly updated client. I've got an AMD client offline right now I'll need to twiddle anyway so that will be a good chance to try it.

[FuzzyNutz]

Are you using filters? I've seen the issue with QTGMC.
I'm not having that issue with AVC/HEVC source either without filters or with LSFmod or FastLineDarkenMod.
I need to test with more filters besides QTGMC.


No filters. The occurrences can make up less than 1% of a movie's total timeline. Unless you've watched every second of every video you've produced, you can't be certain it's not happening to you, too.

[duffbeer]

Please can we get rid of LSmash. It's causing sooooo many problems. Nothing wrong before the update.

[byteshare]

I don't modify the commandline for launch at all. I do use port 1000, low priority, and for restart I use the GUI buttons from the encoding client but generally leave restart alone except when I'm encoding something with KNLMeansCL since this only ever encodes blocks on my host machine and not clients. At least two clients have OpenCL1.2 capable cards installed and configured but reject blocks anyway. The videos encode on those machines standalone fine though :confused:
If you have time please consider testing since if the dev doesn't get help testing on other systems they won't be able to fix problems they can't find. For example, I'm not having those issues so I couldn't give the dev feedback.

No filters. The occurrences can make up less than 1% of a movie's total timeline. Unless you've watched every second of every video you've produced, you can't be certain it's not happening to you, too.
Fair, of the 12+ encodes I've watched completely (different sources) and the ~100 or so I've skipped through I haven't seen an issue like that. I did find a source that had the audio/video out of sync but muxing in the time codes fixed that.

Please can we get rid of LSmash. It's causing sooooo many problems. Nothing wrong before the update.
It was to fix some issues with AC1 I think but other programs like staxrip uses lsmash so it isn't an issue by its self. Personally, all of my initial issues have been ironed out.

[stax76]

It was to fix some issues with AC1 I think but other programs like staxrip uses lsmash so it isn't an issue by its self. Personally, all of my initial issues have been ironed out.

StaxRip use this defaults:

If Check(AviSynthFilterPreferences, "AviSynth Source Filter Preferences", 3) Then
AviSynthFilterPreferences = New StringPairList
AviSynthFilterPreferences.Add("default", "FFVideoSource")
AviSynthFilterPreferences.Add("264 h264 avc", "LWLibavVideoSource")
AviSynthFilterPreferences.Add("265 h265 hevc hvc", "LWLibavVideoSource")
AviSynthFilterPreferences.Add("d2v", "MPEG2Source")
AviSynthFilterPreferences.Add("dgi", "DGSource")
AviSynthFilterPreferences.Add("dgim", "DGSourceIM")
AviSynthFilterPreferences.Add("mp4 m4v mov", "LSMASHVideoSource")
AviSynthFilterPreferences.Add("ts m2ts mts m2t", "LWLibavVideoSource")
AviSynthFilterPreferences.Add("wmv", "DSS2")
AviSynthFilterPreferences.Add("vdr", "AviSource")
End If

If Check(VapourSynthFilterPreferences, "VapourSynth Source Filter Preference", 5) Then
VapourSynthFilterPreferences = New StringPairList
VapourSynthFilterPreferences.Add("default", "ffms2")
VapourSynthFilterPreferences.Add("264 h264 avc", "LWLibavSource")
VapourSynthFilterPreferences.Add("265 h265 hevc hvc", "LWLibavSource")
VapourSynthFilterPreferences.Add("avs vdr", "AVISource")
VapourSynthFilterPreferences.Add("mp4 m4v mov", "LibavSMASHSource")
VapourSynthFilterPreferences.Add("ts m2ts mts m2t", "LWLibavSource")
VapourSynthFilterPreferences.Add("d2v", "d2vsource")
VapourSynthFilterPreferences.Add("dgi", "DGSource")
End If

The default for mkv is still ffms2!

[userx]

Are you able to browse to the share and see the file? Could be an issue with the share.

After last update it seems to work again but EServer take looooong time to create lwi file.
Before this changes, the encoding would have been finished in the meantime.
My setup consists of 5 local server and 3 remote server. On both machines the CPU isn't on full load (~50%) during 5 simultan lwi-jobs. It looks like this changes significant slows down the whole process. 1h45 instead of ~35 min

[ReinerSchweinlin]

Your post will be forgoten because approval may take weeks.
ah ok, thank you, that makes sense. When I posted them, I was able to see them, so I didn't notice. Will put them somewhere "safe" externaly...

[ReinerSchweinlin]

After last update it seems to work again but EServer take looooong time to create lwi file.
Before this changes, the encoding would have been finished in the meantime.
My setup consists of 5 local server and 3 remote server. On both machines the CPU isn't on full load (~50%) during 5 simultan lwi-jobs. It looks like this changes significant slows down the whole process.

Have you tried closing the server window while the LWI file is created?

[userx]

Have you tried closing the server window while the LWI file is created?

close=kill server or minimize? What would be the difference?
without server the jop won't run. (?)

[ReinerSchweinlin]

close=kill server or minimize? What would be the difference?
without server the jop won't run. (?)
ah yes.. minimize :)

[byteshare]

StaxRip use this defaults:

If Check(AviSynthFilterPreferences, "AviSynth Source Filter Preferences", 3) Then
AviSynthFilterPreferences = New StringPairList
AviSynthFilterPreferences.Add("default", "FFVideoSource")
AviSynthFilterPreferences.Add("264 h264 avc", "LWLibavVideoSource")
AviSynthFilterPreferences.Add("265 h265 hevc hvc", "LWLibavVideoSource")
AviSynthFilterPreferences.Add("d2v", "MPEG2Source")
AviSynthFilterPreferences.Add("dgi", "DGSource")
AviSynthFilterPreferences.Add("dgim", "DGSourceIM")
AviSynthFilterPreferences.Add("mp4 m4v mov", "LSMASHVideoSource")
AviSynthFilterPreferences.Add("ts m2ts mts m2t", "LWLibavVideoSource")
AviSynthFilterPreferences.Add("wmv", "DSS2")
AviSynthFilterPreferences.Add("vdr", "AviSource")
End If
Nice find but I wonder why they only use LSmash with mp4, m4v, and mov?

After last update it seems to work again but EServer take looooong time to create lwi file.
Before this changes, the encoding would have been finished in the meantime.
My setup consists of 5 local server and 3 remote server. On both machines the CPU isn't on full load (~50%) during 5 simultan lwi-jobs. It looks like this changes significant slows down the whole process. 1h45 instead of ~35 min
Have you tried updating and clearing out your temp jobs and re-adding jobs?
I haven't noticed that issue with the most recent version. Does it only happen when the chunks are being created when you first start a new job and once it gets going it is fine?
I keep my encoding server window minimized but haven't seen that issue.
I just tried starting a fresh new job and now I see what you're saying...I guess I haven't monitored that part as closely.

[BLKMGK]

<snip>
I just tried starting a fresh new job and now I see what you're saying...I guess I haven't monitored that part as closely.

Does it create a new index after every chunk? On each client? I'll try it again here after a few movies have been completed (single machine only since I'm filtering) and the deck is cleared but index creation previous killed really my speeds.:eek: A single creation of an index is no biggie and expected but not over and over...

[userx]

ah yes.. minimize :)

It doesn't differ. The lwi-process is pretty slow. Why it's switch to this mechanism? What are the benefits? Is there a possibility to switch back to the 'old' bahaviour?

Nice find but I wonder why they only use LSmash with mp4, m4v, and mov?
Have you tried updating and clearing out your temp jobs and re-adding jobs?
I haven't noticed that issue with the most recent version. Does it only happen when the chunks are being created when you first start a new job and once it gets going it is fine?
I keep my encoding server window minimized but haven't seen that issue.
I just tried starting a fresh new job and now I see what you're saying...I guess I haven't monitored that part as closely.

It's a fresh created Job, so I don't have re-added and cleaned someting.
Every chunk has a (slow) lwi-job before it starts, which slows the whole process.

https://i.imgur.com/LPT6KU6.png
https://i.imgur.com/cPAiAsN.png

[byteshare]

Does it create a new index after every chunk? On each client? I'll try it again here after a few movies have been completed (single machine only since I'm filtering) and the deck is cleared but index creation previous killed really my speeds.:eek: A single creation of an index is no biggie and expected but not over and over...

Yeah, on each chunk

It doesn't differ. The lwi-process is pretty slow. Why it's switch to this mechanism? What are the benefits? Is there a possibility to switch back to the 'old' bahaviour?

It's a fresh created Job, so I don't have re-added and cleaned someting.
Every chunk has a (slow) lwi-job before it starts, which slows the whole process.

the LSmash has some benefits but I thought they were only for a specific codec and not x265/x264

[LigH]

Please can we get rid of LSmash. It's causing sooooo many problems. Nothing wrong before the update.

No. :p Problems are to be solved, not to despair of them. :sly: Once solved, you will love the result. :cool:

[LigH]

Nice find but I wonder why they only use LSmash with mp4, m4v, and mov?

LSMASH*Source works only with containers compatible to the ISO base media file format (https://en.wikipedia.org/wiki/ISO_base_media_file_format), but relies on their internal keyframe index chunk and doesn't create a separate index file.

For all other containers, you need LwLibav*Source which is much more flexible, but creates an additional index file to be able.

[byteshare]

LSMASH*Source works only with containers compatible to the ISO base media file format (https://en.wikipedia.org/wiki/ISO_base_media_file_format), but relies on their internal keyframe index chunk and doesn't create a separate index file.

For all other containers, you need LwLibav*Source which is much more flexible, but creates an additional index file to be able.
Ah, okay. If we're already creating a .lwi from the source video (the whole thing) is there any reason we should need to create another .lwi file for chunks of the same video rather than just use the main .lwi file at different points in the file?
Or is there a reason there is an advantage to doing it that way?
Just thinking if we have to create individual .lwi files for DE mode might as well skip that until we're encoding if the source.lwi file isn't going to be used.

No. :p Problems are to be solved, not to despair of them. :sly: Once solved, you will love the result. :cool:
Good point

Hey.
Thank you for the offer. I had a closer look into the temp folders... The two audio streams are extracted, audiolist.txt lists two streams and audiostreams.cmd lists to prozess 3 streams.
What´s missing is content in "encodeaudio2.cmd" - it has zero bytes.
I've been looking at this more (batching more than 1 audio stream & more than 1 sub).
It would be nice if the batch mode could handle more than one stream because I was looking to simplify my process rather than having so many steps.
-Main reason for muxing subs from source with a batch file is so I can keep the MKV title info but mostly for anime since that tends to have the most important context info.
-I wouldn't mind using the internal RipBot audio encoders for items with more than one audio track if they were accounted for. I was realizing I was doing all of my sources' audio externally just for consistency for when I need to run a batch file to mux things back together.

[george84]

RipBot264 newest version on Windows 10.

When I exit RipBot264 then it still exists in list of task manager. I have to abort it to make a restart.

When I want to Minimize, then I can only do so in main menu at upper left of window. That means I can click Minimize and it disappears. It still exists in task manager but with CTRL/Tab I can't get it back.

I have a 4K screen.

[byteshare]

RipBot264 newest version on Windows 10.

When I exit RipBot264 then it still exists in list of task manager. I have to abort it to make a restart.

When I want to Minimize, then I can only do so in main menu at upper left of window. That means I can click Minimize and it disappears. It still exists in task manager but with CTRL/Tab I can't get it back.

I have a 4K screen.

Are you running Logitech Software. See post here: https://forum.doom9.org/showpost.php?p=1879997&postcount=17029

Alternatively update Logitech Software here: https://forum.doom9.org/showpost.php?p=1880045&postcount=17034

Hope this helps..

[slalom]

So, after I did that
I downloaded and extracted the files from the first page
I ran that to update to the latest files, checked the files with yours
I removed all jobs from previous version
I copied the updated folder everywere and added 5 jobs

Same thing, massive network usage
I've gone back to v1.24.0 for a couple of days

Then I read this post
It was to fix some issues with AC1 I think but other programs like staxrip uses lsmash so it isn't an issue by its self. Personally, all of my initial issues have been ironed out.
and I thought maybe I shouldn't have copied the old ini files

So, I re-extracted the zip from the first page and updated it. Changed my settings and added 3 jobs.
Now it looks waaaay faster than before, but

I used my stopwatch to count the time between finishing and re-starting of a server (the next chunk).

That was just over 5 minutes, and the network usage to the roof, again
Anyone else has that problem?

[george84]

Are you running Logitech Software. See post here: https://forum.doom9.org/showpost.php?p=1879997&postcount=17029

Alternatively update Logitech Software here: https://forum.doom9.org/showpost.php?p=1880045&postcount=17034

Hope this helps..

Thank you. I don't have Logitech, but Wacom Tablet software installed. However table is not in use. After aborting these processes, RipBot264 will do an exit when requested.

I hope that a Wacom driver update will correct this.

[slalom]

Ok, second job is going faster, just a few seconds between chunks, don't know why

First two jobs completed with an error. Combining chunks didn't run

Error: The file 'E:\Temp\RipBot264temp\video.264' could not be opened for reading: open file error.

But I can manually run CombineAllChunks.cmd & jobX_MuxFiles.cmd and get my file

[nekrosoft13]

Hey Atak

About week (maybe two) there was a update that is started causing weird issues on my system.

I'm using DE.

I don't know how to describe the, issue, in few places in a file, the video turns in a flashing slideshow with bunch of random frames from previous scenes. Its weird.

here is the sample

https://www.mediafire.com/file/woqlu9wmfw2ufnb/sample.mkv/file

[byteshare]

Ok, second job is going faster, just a few seconds between chunks, don't know why

First two jobs completed with an error. Combining chunks didn't run

Error: The file 'E:\Temp\RipBot264temp\video.264' could not be opened for reading: open file error.

But I can manually run CombineAllChunks.cmd & jobX_MuxFiles.cmd and get my file

What is the source filename exactly?
I recall having that issue with some characters in a filename.

Hey Atak

About week (maybe two) there was a update that is started causing weird issues on my system.

I'm using DE.

I don't know how to describe the, issue, in few places in a file, the video turns in a flashing slideshow with bunch of random frames from previous scenes. Its weird.

here is the sample

https://www.mediafire.com/file/woqlu9wmfw2ufnb/sample.mkv/file

That looks like the same issue I was having with a few files. I don't really understand what is causing it since not all files have that issue.
I noticed it with a new file. Going to see if I get the same issue rerunning the file as I tried encoding just the part with issues in isolation and didn't have an issue. The video had frames randomly from the end of the file well before the end of the video.
Here is the source (00), the encode with the issue (01), and the re-encode from the source once I cut it down to the 8s clip (02): https://mega.nz/#F!UhIHTSzK!WziTOfYZYG80sLYHlu-DHg
I've been going through and not every file has this issue so I'm not sure what is causing it. Seems to be very common on my videos over 30min.
At this point encoding doesn't seem stable to me.

[george84]

Thank you. I don't have Logitech, but Wacom Tablet software installed. However table is not in use. After aborting these processes, RipBot264 will do an exit when requested.

I hope that a Wacom driver update will correct this.

Unfortunately this didn't help. Temporary solution remains to abort Wacom processes or prevent their autostart.

[slalom]

What is the source filename exactly?
I recall having that issue with some characters in a filename.
No special characters on the filename.
Anyway I closed and started Ripbot and the next encode completed successfully

I need to check network usage today. I suppose the usual are short spikes about 30% network usage when completing chunks, but I saw higher and longer usage at some point. I need to see that again today

[duffbeer]

Hey Atak

About week (maybe two) there was a update that is started causing weird issues on my system.

I'm using DE.

I don't know how to describe the, issue, in few places in a file, the video turns in a flashing slideshow with bunch of random frames from previous scenes. Its weird.

here is the sample

https://www.mediafire.com/file/woqlu9wmfw2ufnb/sample.mkv/file

I have had the same problem too. Seems like some files have frames that jump around like your example and some files have kind of the opposite, a very slow choppy playback at about 2fps.
I've gone back to 1.24.1 and everything is spot on. I don't see why we needed LSmash when everything was working fine with FFMS.
There seems to be no advantage to LSmash so why go through all the problems??

[slalom]

The normal is on the left

https://i.ibb.co/3kPcXJg/image.jpg (https://imgbb.com/)

[Atak_Snajpera]

Regarding flashing frames. What codec is used in source file? Mpeg2,avc,hevc?

[nekrosoft13]

Regarding flashing frames. What codec is used in source file? Mpeg2,avc,hevc?

it doesn't happen in short files, 20-30 minutes

it happens a lot in longer files.

it all started about 1-2 weeks ago

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 3 Ref Frames
Format settings, CABAC : Yes
Format settings, ReFrames : 3 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 1 h 26 min
Bit rate mode : Variable
Bit rate : 5 150 kb/s
Maximum bit rate : 25.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.099
Stream size : 3.04 GiB (87%)
Writing library : x264 core 135 r2345 f0c1c53
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=5150 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=25000 / vbv_bufsize=25000 / nal_hrd=vbr / ip_ratio=1.40 / aq=1:1.00
Default : Yes
Forced : No

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 47 min 12 s
Bit rate mode : Constant
Bit rate : 9 322 kb/s
Nominal bit rate : 10 000 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.188
Stream size : 3.07 GiB (94%)
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

[byteshare]

Regarding flashing frames. What codec is used in source file? Mpeg2,avc,hevc?
Source is x264 (AVC). I saved 6 files that I'm testing on now. 5 are AVC L4.0 and one L4.2
I added the media info files here:
https://mega.nz/#F!UhIHTSzK!WziTOfYZYG80sLYHlu-DHg
So far I've only managed to get through one file and it repeated the issue with the frames.
---Update: I also confirmed this issue with another AVC L4.0 file NOT using DE mode, so it doesn't seem exclusive to DE mode.
...besides the flashing frames the slowness of creating index files for every chunk is a major issue since it makes some encodes overall very, very slow even when the actual encode times are fast. Is there a way to speed this up by using the .lwi file we create when we're adding jobs?

it doesn't happen in short files, 20-30 minutes

it happens a lot in longer files.

it all started about 1-2 weeks ago
You were able to confirm this too? At least we're narrowing it down.

The normal is on the left

https://i.ibb.co/3kPcXJg/image.jpg (https://imgbb.com/)
Network usage, yes? At what point is that happening for you? I have two encoding servers but they are local so that might be why I'm not seeing those spikes?

[byteshare]

Just noticed the update from 8/20. Updated core from 8.17 to 8.20 (please consider changing the release number in the main window: 1.25.x)
Re-added the problem source videos as jobs.
Realized that my encoding servers never started.
I see the "Select SuperviseProcess.exe
In the window I see this:
[2019-08-22 11:21:04] C:\RipBot\EncodingServer.exe /start /restart-if-no-progress /minimize /ip 0.0.0.0 /port 2000 (PID:3268) executed.
[2019-08-22 11:21:05] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:06] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:07] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:08] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:09] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:10] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:11] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:12] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:13] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:14] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:15] EncodingServer.exe (PID:3268) is NOT responding.
...etc until the server restarts and keeps looping like that.
I've tried a reboot. I've tried killing my startup processes, I've tried closing everything I know to close.
I can't get the encoding server to start.
Non-DE mode starts and will try that in a bit. Running a StaxRip test since it uses the same filter to see if I get the flicker issue.
Update: Stax only got 20% through the file before giving up. RB in non-DE mode is still going and over 50% through.

[slalom]

Network usage, yes? At what point is that happening for you? I have two encoding servers but they are local so that might be why I'm not seeing those spikes?
I think it's random. I have four encoding servers

[nekrosoft13]

Source is x264 (AVC). I saved 6 files that I'm testing on now. 5 are AVC L4.0 and one L4.2
I added the media info files here:
https://mega.nz/#F!UhIHTSzK!WziTOfYZYG80sLYHlu-DHg
So far I've only managed to get through one file and it repeated the issue with the frames.
---Update: I also confirmed this issue with another AVC L4.0 file NOT using DE mode, so it doesn't seem exclusive to DE mode.
...besides the flashing frames the slowness of creating index files for every chunk is a major issue since it makes some encodes overall very, very slow even when the actual encode times are fast. Is there a way to speed this up by using the .lwi file we create when we're adding jobs?


You were able to confirm this too? At least we're narrowing it down.


Network usage, yes? At what point is that happening for you? I have two encoding servers but they are local so that might be why I'm not seeing those spikes?

Yes I was able to confirm it.

[FuzzyNutz]

Are you running Logitech Software. See post here: https://forum.doom9.org/showpost.php?p=1879997&postcount=17029

Alternatively update Logitech Software here: https://forum.doom9.org/showpost.php?p=1880045&postcount=17034

Hope this helps..

This worked for me. Starting in December of 2018, RB would not close cleanly. Today I exited my Logitech gaming software, which I use for my G602 mouse and usually have running in the background, to my amazement, RB closes cleanly. I've downloaded the recommended alternative software "G Hub", but have not tried it yet. Fortunately, the problematic software, "Gaming Software", does not need to run in the background, cuz the settings are saved to my mouse's internal memory.

Thank you.

[FuzzyNutz]

Regarding flashing frames. What codec is used in source file? Mpeg2,avc,hevc?

AVC for some; maybe all. (1080p 23.976fps H264 blu-ray sources)

[FuzzyNutz]

Can a modern solid-state drive endure daily use of blu-ray size projects with RipBot264? If so, it would be a way to reduce wait times for demuxing/muxing and job loading.

[byteshare]

Can a modern solid-state drive endure daily use of blu-ray size projects with RipBot264? If so, it would be a way to reduce wait times for demuxing/muxing and job loading.
I use an SSD for my temp drive and for remuxing files afterwards with audio/subs. So far put over 20TB of writes on it in about a year and it is doing well.
For sure speeds up some of the work.

[byteshare]

Just noticed the update from 8/20. Updated core from 8.17 to 8.20 (please consider changing the release number in the main window: 1.25.x)
Re-added the problem source videos as jobs.
Realized that my encoding servers never started.
I see the "Select SuperviseProcess.exe
In the window I see this:
[2019-08-22 11:21:04]
C:\RipBot\EncodingServer.exe /start /restart-if-no-progress /minimize /ip 0.0.0.0 /port 2000
(PID:3268) executed.
[2019-08-22 11:21:05] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:06] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:07] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:08] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:09] EncodingServer.exe (PID:3268) is responding.
[2019-08-22 11:21:10] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:11] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:12] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:13] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:14] EncodingServer.exe (PID:3268) is NOT responding.
[2019-08-22 11:21:15] EncodingServer.exe (PID:3268) is NOT responding.
...etc until the server restarts and keeps looping like that.
I've tried a reboot. I've tried killing my startup processes, I've tried closing everything I know to close.
I can't get the encoding server to start.
Non-DE mode starts and will try that in a bit. Running a StaxRip test since it uses the same filter to see if I get the flicker issue.
Update: Stax only got 20% through the file before giving up. RB in non-DE mode is still going and over 50% through.
Update:
- Okay in DE mode & non-DE mode, I got the flickering in Core 2019.08.17.
- StaxRip 2.0.3 doesn't seem to have the issue but playback (FPS) is wrong and it can't complete the whole file (~20% of the total frames)
+/- Non-DE mode (since encoding servers won't start for me) I'm not seeing the flickering in Core 2019.08.20...so it might be fixed (need more testing to feel confident)
+ In Handbrake I don't have the issue.

If Core 2019.08.20 is fixed that is great. Not sure if the index slowness issues was fixed as well since I can't get the encoding server to start.

Update-Update....I don't know why but the servers are starting now...I'll try DE mode in Core 2019.08.20
-NOT seeing the lwi indexes being created for chunks! :D

[FuzzyNutz]

I use an SSD for my temp drive and for remuxing files afterwards with audio/subs. So far put over 20TB of writes on it in about a year and it is doing well.
For sure speeds up some of the work.

What SSD are you using? I'm considering either a 2 or 4 Terabyte SSD to increase longevity.

[LigH]

Video conversion will mainly write large files with linear access. An SSD should like this kind of access with little wear.

[FuzzyNutz]

Video conversion will mainly write large files with linear access. An SSD should like this kind of access with little wear.

I hope you're correct. I'm concerned that the large amount of writing would reduce a SSD's lifespan faster than typical usage.

[Atak_Snajpera]

I hope you're correct. I'm concerned that the large amount of writing would reduce a SSD's lifespan faster than typical usage.

Typical MLC SSD has 3000 P/E cycles. After that number data retention in room temperature drops below 1 year.
https://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention

I have already used all 3000 P/E cycles and my SSD still works
https://i.imgsafe.org/fe/fe6dee4d4d.png

[FuzzyNutz]

Update:
- Okay in DE mode & non-DE mode, I got the flickering in Core 2019.08.17.
- StaxRip 2.0.3 doesn't seem to have the issue but playback (FPS) is wrong and it can't complete the whole file (~20% of the total frames)
+/- Non-DE mode (since encoding servers won't start for me) I'm not seeing the flickering in Core 2019.08.20...so it might be fixed (need more testing to feel confident)
+ In Handbrake I don't have the issue.

If Core 2019.08.20 is fixed that is great. Not sure if the index slowness issues was fixed as well since I can't get the encoding server to start.

Update-Update....I don't know why but the servers are starting now...I'll try DE mode in Core 2019.08.20
-NOT seeing the lwi indexes being created for chunks! :D

My RB core date is 2019-08-17 and the flickering is still happening.

[FuzzyNutz]

Typical MLC SSD has 3000 P/E cycles. After that number data retention in room temperature drops below 1 year.
https://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention

I have already used all 3000 P/E cycles and my SSD still works
https://i.imgsafe.org/fe/fe6dee4d4d.png

My rig runs 24/7. I move completed RipBot264 projects to mechanical hard drives. Retention when my system isn't running was not my focus, but thanks for the info.
I used the formula found here to estimate lifespan based on write usage: https://www.compuram.de/blog/en/the-life-span-of-a-ssd-how-long-does-it-last-and-what-can-be-done-to-take-care/

[byteshare]

My RB core date is 2019-08-17 and the flickering is still happening.
Update! At the very least 8.20 Core doesn't have the index rebuild issue for each chunk. I was going to run more test but fell asleep. Doing that now. At least 1 of 6 didn't get the flickering when on 8.17 it was getting it each time (ran test 3 times).

What SSD are you using? I'm considering either a 2 or 4 Terabyte SSD to increase longevity.
Correction, 30TB written. I have 4 different SSDs but I recommend the Samsung EVO line since they have one of the best warranties if you're really worried. I still use a physical drive for storage of source files.

[FuzzyNutz]

Update! At the very least 8.20 Core doesn't have the index rebuild issue for each chunk. I was going to run more test but fell asleep. Doing that now. At least 1 of 6 didn't get the flickering when on 8.17 it was getting it each time (ran test 3 times).


Correction, 30TB written. I have 4 different SSDs but I recommend the Samsung EVO line since they have one of the best warranties if you're really worried. I still use a physical drive for storage of source files.

I've tried deleting the "updater" and "update_log" files with hopes of nudging to the newest available update, but my core date is still 08-17 according to Windows Explorer. It shows as version 1.25.0.0.
When and if I do get RB to update the core beyond 08-17, will I need to reload projects that were loaded while the core was 08-17?

My OS and other software are on a 500G 970 EVO Plus NVMe M.2 and my RipBot264 projects are on an HDD. I'm considering using an 860 EVO 2.5" 4TB SATA III for RipBot264 projects. My completed RipBot264 projects are on HDD's.

[brumsky]

I've tried deleting the "updater" and "update_log" files with hopes of nudging to the newest available update, but my core date is still 08-17.
When and if I do get RB to update the core beyond 08-17, will I need to reload projects that were loaded while the core was 08-17?

My OS and other software are on a 500G 970 EVO Plus NVMe M.2 and my RipBot264 projects are on an HDD. I'm considering using an 860 EVO 2.5" 4TB SATA III for RipBot264 projects. My completed RipBot264 projects are on HDD's.

I have several SSDs that are very old, one is an OCZ Vertex2 which came out 6+ years ago. It was back before SSDs were maxing out the SATAIII bus - that just shows you how old it really is!

https://imgur.com/rw39jSj

Both of the 850 evos are 250GB SSDs. I used them for a year or two to run a DB that had a ton of writes. Then for past few years I've been using them as a temp dir for ripbot and Plex DB.

The OCZ drive is still going strong. it is currently my ISO store on a PVE cluster. It just won't die... lol

I really want to get a MyDigitalSSD BPX Pro. The 1TB version is warrantied for 1,660TBW. Yes you read that right 1.6PBs of writes. It normally sells for 110-120 USD. They are hard to find now though, my guess is they are going to release a new SSD line soon. the BPX Pro isn't the fastest but you have to love that TBW warranty!!


EDIT:
By the way everything is working perfect now!! Thank you all for the hard work to fix this!

[FuzzyNutz]

I've tried deleting the "updater" and "update_log" files with hopes of nudging to the newest available update, but my core date is still 08-17 according to Windows Explorer. It shows as version 1.25.0.0.
When and if I do get RB to update the core beyond 08-17, will I need to reload projects that were loaded while the core was 08-17?

How can I force RB to update to the newest version?

[byteshare]

How can I force RB to update to the newest version?
Edit the update.ini in the RipBot folder. Change the LastCheck= to something from last year, save, reopen RB, check the Updates folder and you should be seeing new files come in. I don't know the threshold but that works for me.

[FuzzyNutz]

Edit the update.ini in the RipBot folder. Change the LastCheck= to something from last year, save, reopen RB, check the Updates folder and you should be seeing new files come in. I don't know the threshold but that works for me.

Didn't work for me.

[nekrosoft13]

Edit the update.ini in the RipBot folder. Change the LastCheck= to something from last year, save, reopen RB, check the Updates folder and you should be seeing new files come in. I don't know the threshold but that works for me.

doesn't work

[nekrosoft13]

Typical MLC SSD has 3000 P/E cycles. After that number data retention in room temperature drops below 1 year.
https://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention

I have already used all 3000 P/E cycles and my SSD still works
https://i.imgsafe.org/fe/fe6dee4d4d.png

Goodram, i see you buying local ;)

[nekrosoft13]

I hope you're correct. I'm concerned that the large amount of writing would reduce a SSD's lifespan faster than typical usage.

try to stick with MLC and TLC (at the worse) avoid the new cheap QLC drives.

QLC are becoming really popular due to cheap prices, which soon could be a problem and MLC and TLC could disapear from consumer products.

In eyes of consumer prices matters the most.

MLC right now is pretty much enterprise only.

[Atak_Snajpera]

Yeah! Avoid QLC like plague! They are very slow outside SLC cache (less than 80MiB/s!) and also P/E is around 100.
https://hardforum.com/threads/crucial-p1-1tb-brutally-slow-and-hot.1982110/

[byteshare]

Didn't work for me.
doesn't work
Huh, what I just did and I see another update today to the EncodingClient, EncodingServer, RipBot264, and update exes
Not sure what changed though...
Update: oh, new options in settings for default decoder (LSMASH or FFMS2)
Thank you for the update and being a great Dev.

[FuzzyNutz]

Huh, what I just did and I see another update today to the EncodingClient, EncodingServer, RipBot264, and update exes
Not sure what changed though...
Update: oh, new options in settings for default decoder (LSMASH or FFMS2)

I can't get RB to update on demand. Deleting the "update_log" and/or "updater" files won't work. Changing the "lastcheck" to an older date in the "updater" file won't work. Replacing the core to 1.24 won't work. Run RB as administrator won't work. I have "Use Auto-update" checked in the RB Advanced Settings. RB updates when it feels like it. Grrrr.

[byteshare]

I can't get RB to update on demand. Deleting the "update_log" and/or "updater" files won't work. Changing the "lastcheck" to an older date in the "updater" file won't work. Replacing the core to 1.24 won't work. Run RB as administrator won't work. I have "Use Auto-update" checked in the RB Advanced Settings. RB updates when it feels like it. Grrrr.
First, you should already be running RipBot as admin for it to work right in most cases.
I've been manually changing the last LastCheck in the updater.ini for years.
Try this:
Confirmed "Use Auto-update" is checked > Close RipBot > Open the updater.ini in notpad > Change the LastCheck from something like "LastCheck=2019-08-23 10:06:22" to "LastCheck=2016-08-23 10:06:22" > Save the file > open RipBot as admin > Check the update folder: \RipBot\Updates > once no new files come and and the files are there are no longer growing in size > Close RipBot > Open RipBot as admin and the update should start to install.
You might need to repeat those steps a few times to get completely up to date depending on your current version.

The above method does work, as I've used it for years on more than one machine. So, if it is not working for you than something is either being done wrong or something else is wrong on your setup.

If you are not seeing an update file quickly being made in the folder when you first open RipBot check that your settings are actually correct.

If you are not on Core 08.23 and you are not seeing files be downloaded you might check your firewall settings for RipBot (ie non-local connections are not blocked).

[FuzzyNutz]

First, you should already be running RipBot as admin for it to work right in most cases.
I've been manually changing the last LastCheck in the updater.ini for years.
Try this:
Confirmed "Use Auto-update" is checked > Close RipBot > Open the updater.ini in notpad > Change the LastCheck from something like "LastCheck=2019-08-23 10:06:22" to "LastCheck=2016-08-23 10:06:22" > Save the file > open RipBot as admin > Check the update folder: \RipBot\Updates > once no new files come and and the files are there are no longer growing in size > Close RipBot > Open RipBot as admin and the update should start to install.
You might need to repeat those steps a few times to get completely up to date depending on your current version.

The above method does work, as I've used it for years on more than one machine. So, if it is not working for you than something is either being done wrong or something else is wrong on your setup.

If you are not seeing an update file quickly being made in the folder when you first open RipBot check that your settings are actually correct.

If you are not on Core 08.23 and you are not seeing files be downloaded you might check your firewall settings for RipBot (ie non-local connections are not blocked).

What is the latest version of the core? I have 1.25.0.0.

[byteshare]

What is the latest version of the core? I have 1.25.0.0.
Core 2019.08.23 You have to look in the Settings > Tools > Installed Tools > Core
The RipBot version in the main window doesn't change with updates always, so it will still read 1.25.0
Personally the Window name should just be RipBotx264
and the version should be pulled from something else and shown in the main GUI, so that when updates are pushed out only changing the updater version or an .ini file would be required rather than recompiling GUI or something like that so even when filters are updated we'd see a version change.

[FuzzyNutz]

Core 2019.08.23 You have to look in the Settings > Tools > Installed Tools > Core
The RipBot version in the main window doesn't change with updates always, so it will still read 1.25.0
Personally the Window name should just be RipBotx264
and the version should be pulled from something else and shown in the main GUI, so that when updates are pushed out only changing the updater version or an .ini file would be required rather than recompiling GUI or something like that so even when filters are updated we'd see a version change.

I'm up to date. Thanks.
I'll chime in once I know if the frame flickering seems resolved.

[FuzzyNutz]

I'm up to date. Thanks.
I'll chime in once I know if the frame flickering seems resolved.

I'm still getting the flickering with core 08-23.

[byteshare]

I'm still getting the flickering with core 08-23.
Huh...I'm not, but you can switch back to FFMS2 if you weren't having an issue with that one.
Since 8.20 I wasn't having the issues since the Encoding Servers stopped trying to recreate the index files...at least on the 6 files that I had flickering between the 2-4 encodes for each with versions since the LSmash addition but before 8.20.

[FuzzyNutz]

Huh...I'm not, but you can switch back to FFMS2 if you weren't having an issue with that one.
Since 8.20 I wasn't having the issues since the Encoding Servers stopped trying to recreate the index files...at least on the 6 files that I had flickering between the 2-4 encodes for each with versions since the LSmash addition but before 8.20.

I'm not using DE. What's FFMS2?

[byteshare]

I'm not using DE. What's FFMS2?
Using DE mode doesn't matter, since 8.20 the way indexing works is different. I confirmed the flickering in DE and non-DE and then it working in both.
FFMS2 is the way before LSmash. You can change the default decoder in the settings from LSmash to FFMS2 for all codecs or just the ones you want.
Even though LSmash is working I switched back to FFMS2 now that we have the option since creating jobs is faster and I wasn't having issues before LSmash.

[FuzzyNutz]

Using DE mode doesn't matter, since 8.20 the way indexing works is different. I confirmed the flickering in DE and non-DE and then it working in both.
FFMS2 is the way before LSmash. You can change the default decoder in the settings from LSmash to FFMS2 for all codecs or just the ones you want.
Even though LSmash is working I switched back to FFMS2 now that we have the option since creating jobs is faster and I wasn't having issues before LSmash.

Where is the LSmash/FFMS2 setting? What did you confirm about the flickering?

[byteshare]

Where is the LSmash/FFMS2 setting? What did you confirm about the flickering?
You can change the default decoder in the settings (Main) from LSmash to FFMS2 for all codecs or just the ones you want, or you can even do it after the fast if you right click on a job and select FFMS2.
I confirmed it by going through both the parts I had marked as consistently having issues and skipping through 5s at a time on 6 files that had issues on every encode I did on them (2-4 times depending on the file).

[nekrosoft13]

Using DE mode doesn't matter, since 8.20 the way indexing works is different. I confirmed the flickering in DE and non-DE and then it working in both.
FFMS2 is the way before LSmash. You can change the default decoder in the settings from LSmash to FFMS2 for all codecs or just the ones you want.
Even though LSmash is working I switched back to FFMS2 now that we have the option since creating jobs is faster and I wasn't having issues before LSmash.

switched to FFMS2 as well, will see if it fixes the flickering.

[FuzzyNutz]

You can change the default decoder in the settings (Main) from LSmash to FFMS2 for all codecs or just the ones you want, or you can even do it after the fast if you right click on a job and select FFMS2.
I confirmed it by going through both the parts I had marked as consistently having issues and skipping through 5s at a time on 6 files that had issues on every encode I did on them (2-4 times depending on the file).

No LSmash/FFMS2 setting exists on my UI, "main".
What is the "fast"?
What is the "it" in "I confirmed it"?

[FuzzyNutz]

An error appears in every activation cycle in my update log:

2019-08-24 12:55:57 : =========================[UPDATER ACTIVATED]=========================
2019-08-24 12:55:57 : Looking for correct UUID link in
2019-08-24 12:55:57 : [ERROR] NO CONNECTION with
2019-08-24 12:55:57 : Searching for UUID link with DuckDuckGo...
2019-08-24 12:55:57 : Search results...
2019-08-24 12:55:57 : http://pastebin.com/FC9zCf0D
2019-08-24 12:55:57 : Looking for correct UUID link in http://pastebin.com/FC9zCf0D
2019-08-24 12:55:58 : [SUCCESS] http://pastebin.com/FC9zCf0D has correct UUID link 6c966b28-e0dd-48f6-b1c7-a56e8a275ec0
2019-08-24 12:55:58 : No results.

My Windows 10 (Pro) firewall includes exceptions for RipBot264 and it's updater, that I added.

[slalom]

You can change the default decoder in the settings (Main) from LSmash to FFMS2 for all codecs or just the ones you want, or you can even do it after the fast if you right click on a job and select FFMS2.
I changed everything to FFMS2 too. I hope there are no problems

[Atak_Snajpera]

An error appears in every activation cycle in my update log:

2019-08-24 12:55:57 : =========================[UPDATER ACTIVATED]=========================
2019-08-24 12:55:57 : Looking for correct UUID link in
2019-08-24 12:55:57 : [ERROR] NO CONNECTION with
2019-08-24 12:55:57 : Searching for UUID link with DuckDuckGo...
2019-08-24 12:55:57 : Search results...
2019-08-24 12:55:57 : http://pastebin.com/FC9zCf0D
2019-08-24 12:55:57 : Looking for correct UUID link in http://pastebin.com/FC9zCf0D
2019-08-24 12:55:58 : [SUCCESS] http://pastebin.com/FC9zCf0D has correct UUID link 6c966b28-e0dd-48f6-b1c7-a56e8a275ec0
2019-08-24 12:55:58 : No results.

My Windows 10 (Pro) firewall includes exceptions for RipBot264 and it's updater, that I added.

Replace old updater.exe with this
https://www.mediafire.com/file/0744wilxt2e7wlv/updater.exe/file

[Atak_Snajpera]

I changed everything to FFMS2 too. I hope there are no problems

FFMS2 is not perfect either
https://forum.doom9.org/showthread.php?p=1881599#post1881599

I would at least use LSMASH for MPEG-2. I have few sample where FFMS2 mpeg-2 decoder acts weird.

[FuzzyNutz]

Replace old updater.exe with this
https://www.mediafire.com/file/0744wilxt2e7wlv/updater.exe/file

Thank you.

[slalom]

FFMS2 is not perfect either
https://forum.doom9.org/showthread.php?p=1881599#post1881599

I would at least use LSMASH for MPEG-2. I have few sample where FFMS2 mpeg-2 decoder acts weird.
LSMASH takes at least twice the time. I can use it for MPEG-2, there are very few of them

[byteshare]

FFMS2 is not perfect either
https://forum.doom9.org/showthread.php?p=1881599#post1881599

I would at least use LSMASH for MPEG-2. I have few sample where FFMS2 mpeg-2 decoder acts weird.
Thank you for the note on MPEG-2. I had LSmash only set for VC-1.

No LSmash/FFMS2 setting exists on my UI, "main".
What is the "fast"?
What is the "it" in "I confirmed it"?
You're not on the most recent version if you don't see that in the main section of your settings.
"fast"...*fact (typo)
"it" confirming the issue wasn't happening

[Atak_Snajpera]

Do not use lsmash for vc1!!! IT is totally broken!!!

[FuzzyNutz]

Once RB treats me to the update that adds the setting for selecting LSmash, should I use it for MPEG-2 sources only?

[slalom]

@Atak
I wanted to ask something regarding which server runs the CombineAllChunks.cmd
Is it the one who finishes the last chunk?

[Atak_Snajpera]

None. Encodindclient.exe executes that command file.

[byteshare]

Do not use lsmash for vc1!!! IT is totally broken!!!
Ah, okay...I did NOT remember that correctly. Thank you.

Once RB treats me to the update that adds the setting for selecting LSmash, should I use it for MPEG-2 sources only?
It isn't RB if you've edited the update.ini file, but to answer your question: You can use it for LSmash for all except VC-1 or just MPEG-2, up to you.

[byteshare]

I've been having some slowness issues and I thought it was more to do with LSmash but it doesn't seem to be...
I get about 1FPS using SMD in DE mode (1 encoding server) but about 7 FPS in non-DE mode with the exact same job. My SMD code:video=video.SMDegrain(tr=3,thSAD=400,thSADC=150,refinemotion=false,contrasharp=true,plane=4,pel=2,prefilter=4,
Truemotion=False,chroma=true,hpad=32,vpad=32,str=2,amp=1) (added line-break to help with horizontal scrolling
One thing I've noticed that the encoding server is spending most of the time in decoding mode and doesn't actually encode very much and if I have 2 encoding servers at some point it is common for them both to stall and just do decoding but at a reduced amount of CPU.
I'd post about this before and thought it was to do with QTGMC and SMD not playing well together but when I switch to non-DE mode I'm also seeing an increase in FPS for encoding.

Is there something I could do to help DE mode be faster?
I don't have to be in DE mode but I use it for two reasons to normally get slightly faster FPS (using more CPU) and so if I need to "Pause" during a job I can turn off the encoding server, or even reboot if need be.

[Atak_Snajpera]

Network bottleneck.

[byteshare]

Network bottleneck.
Gigabit LAN but just one Encoding Server and it is local (all on the same machine as the main RipBot), so that seems like an issue with what the Encoding Server is doing, no?
-I don't recall seeing the same issues if I don't use filters, specifically MDegrain, SMDegrain, and QTGMC slower preset.

I'd have to go back and test with an older version of RipBot but at some point this started happening because I remember doing a lot of tests with Prefilter 3 and 4 on SMDegrain (CPU (3) and GPU (4)) and getting roughly the same speeds I'm getting now in non-DE mode that I was getting in DE mode with the older version.

[FuzzyNutz]

It isn't RB if you've edited the update.ini file, but to answer your question: You can use it for LSmash for all except VC-1 or just MPEG-2, up to you.

I have core 08-23 and updater 08-24, yet changing to an older date in the updater.ini file still doesn't get me LSmash settings after many runs of RB as administrator or not. I'm still getting flickering frames.

[Atak_Snajpera]

What does updater log say?

[FuzzyNutz]

What does updater log say?

2019-08-25 13:09:38 : =========================[UPDATER ACTIVATED]=========================
2019-08-25 13:09:38 : Looking for correct UUID link in http://forum.doom9.org/showthread.php?t=127611
2019-08-25 13:09:39 : [SUCCESS] http://forum.doom9.org/showthread.php?t=127611 has correct UUID link 6c966b28-e0dd-48f6-b1c7-a56e8a275ec0
2019-08-25 13:09:39 : Downloading update file http://atak-snajpera.5v.pl/ripbot264update/update.zip
2019-08-25 13:09:39 : [SUCCESS] C:\Program Files (x86)\RipBot264\Updates\update.zip saved!
2019-08-25 13:09:39 : CRC32 value has not changed for [core]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [Aften]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [avs2pipemod]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [avs2yuv]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [AVSMeter]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [BDSup2Sub]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [ChapterXtractor]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [dgindex]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [DownloadPoster]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [eac3to]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [ffmpeg]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [fhgaacenc]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [MediaInfo]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [mkvtoolnix]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [ModifyChapters]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [mp4box]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [mpc]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [opus-tools]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [PgcDemux]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [remuxtool]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [SetACL]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [ssatosrt]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [tsmuxer]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [vjoin]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [VSRip]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [wolcmd]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [x264]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [x265]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [AutoCrop]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [ffms]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [flash3kyuu_deband]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [hqdn3d]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [Scripts]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [TIVTC]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [VSFilter]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [Yadif]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [KNLMeansCL]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [OpenCLinfo]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [masktools]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [mvtools]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [nnedi3]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [DetectBorders]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [AviSynth]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [RawSourcePlus]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [RgTools]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [NicAudio]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [Plugins_JPSDR]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [WaitForProcess]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [avsresize]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [DGTonemap]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [lsmash]. Update is not required.
2019-08-25 13:09:39 : CRC32 value has not changed for [7z]. Update is not required.
2019-08-25 13:09:39 : Downloading finished.

[Atak_Snajpera]

You are fully updated. End of story.

[FuzzyNutz]

You are fully updated. End of story.

Why don't I see the option for LSmash under settings and I don't have the ability to choose it when right-clicking loading jobs as described by "byteshare"?

-And I'm still getting frame flickering.

[Atak_Snajpera]

Don't know.

[FuzzyNutz]

Don't know.

Please upload a screenshot of the LSmash setting.

[slalom]

Why don't I see the option for LSmash under settings and I don't have the ability to choose it when right-clicking loading jobs as described by "byteshare"?

-And I'm still getting frame flickering.
Download the file from the first page and update that

[FuzzyNutz]

Download the file from the first page and update that

It worked. Thanks.

[FuzzyNutz]

I get flickering frames using LSmash on AVC sources, but not with FFSM2.

[BLKMGK]

Upgraded, had to reinstall to get it but I've got it :)

The weird behavior with chunks and indexing is gone - whew! However I'm back to my issues with KNLMeansCL - ugh! At first even my desktop that was able to run this code was giving me errors about not being able to access a "file". I switched settings such that both X264 and KNLMeansCL devices were set to the onboard Intel CPU, much to my surprise both my desktop and a client with a GeForce 680 began working - albeit VERY slow. If I stop the desktop and switch it to use the GTX 1070 for KNLMeansCL then the desktop begins working (about double speed) and the remote client begins failing! It looks like settings on the hosting machine affect clients and the client being set to use it's onboard card aren't obeyed :( Currently the single machine can beat the efforts of both machines working together so I'm leaving it alone (lol) but I'd appreciate some insight.

Setting the X265 accel to "none" results in both failing too... I'm also finding that aborting a job midway through no longer saves the completed blocks - that's a huge bummer! It might be because I'm swapping acceleration devices around? This job is FFMS2 based indexing BTW. I'm going to look for a post you made a bit ago that showed a commandline for specifying OpenCL hardware ID numbers. I'm betting that this isn't being drawn from the local machine doing encoding but from the server handing out the jobs. Maybe manually speccing it will help? On the tasking desktop the hardware ID I want is 2 so if it's sending a hardware ID to the other clients it'll be wrong depending on the machine - the 8700K I'm using as a slave doesn't have good enough video encoding on it's CPU apparently? The 3900X I've got but am not currently using has no video in the CPU so it's OpenCL ID will be 1 as well vs the 2 my tasking machine is using.

EDIT: Manually setting the device ID, to the one already set in RipBot on my slave encoder, got it working - whew! Now I want my 3900X back to fix it too lol. Hopefully this helps someone else out. Is there documentation somewhere on the various commandline switches?

P.S. a bug I think I have nailed down... Whenever you abort a job the next time you start it up it will fail. It fails because the tools folder is erased but not copied. The second time after an abort that you start it will work because it copies the tools over to the folder. So to test this I must abort 2x before I can get a good test.

[Atak_Snajpera]

Hopefully this helps someone else out. Is there documentation somewhere on the various commandline switches?
https://i.imgsafe.org/3b/3b7fc0cfd6.png

[byteshare]

I get flickering frames using LSmash on AVC sources, but not with FFSM2.
Well, at least it works with FFSM2 for you now.

P.S. a bug I think I have nailed down... Whenever you abort a job the next time you start it up it will fail. It fails because the tools folder is erased but not copied. The second time after an abort that you start it will work because it copies the tools over to the folder. So to test this I must abort 2x before I can get a good test.
Oh, nice find. That explains some of the weird behavior I've seen when I was using DE mode, but I'm on non-DE mode right now since I have issues with things running slower in that mode.

[FuzzyNutz]

When was LSmash introduced to RB? -So I know how far back to redo my collection with FFSM2.

[byteshare]

When was LSmash introduced to RB? -So I know how far back to redo my collection with FFSM2.

v1.25 Aug. 1st
https://forum.doom9.org/showpost.php?p=1880784&postcount=17063

[FuzzyNutz]

v1.25 Aug. 1st
https://forum.doom9.org/showpost.php?p=1880784&postcount=17063

Thanks. If only I knew when my RB updated to it. First video I recall exhibiting flickering frames was mid August.

[byteshare]

Thanks. If only I knew when my RB updated to it. First video I recall exhibiting flickering frames was mid August.
I believe the x265 version changed too, so you could tell by that if you're encoding into x265.

[byteshare]

Newer QTGMC v3.358s
Supports KNLMeansCL as a denoiser and 16B
There are some bug fixes as well but from testing I'm not seeing much of a difference if using just using Medium preset.
http://avisynth.nl/index.php/QTGMC

[PLOPZA]

I've noticed that in windows 10, running Ripbot in D.E. mode, that windows built-in antivirus and anti-malware is scanning the chunks as they're sent to each server while using L-Smash. And I switched L-smash to ffms2 looks like my encodes are back to acceptable completion times (120-150 fps first pass, 30-34 fps second pass) across 4 PC's

Edit - I would also note all computers, and the bandwidth usage spikes when intercepted by the scanner, scanner does it's thing and then ripbot shows it's encoding and the bandwidth drops to normal 60-110 megabits per second

[Ryushin]

So I just got hit by the random frames bug and I'm using DE. Seems to happen a couple of seconds after a minute mark for 2-5 seconds. I looked at a 4K and HD encode I did with MDegrain2 and at first glance, it seems fine. I have a 3h:17m source that I ran MDegrain2 on and at the 13 minute mark, random frames occur for about 2-10 seconds. It happens to most of the minute marks after that. Going to run the same source through again without the MDegrain2 and see what happens.

[byteshare]

So I just got hit by the random frames bug and I'm using DE. Seems to happen a couple of seconds after a minute mark for 2-5 seconds. I looked at a 4K and HD encode I did with MDegrain2 and at first glance, it seems fine. I have a 3h:17m source that I ran MDegrain2 on and at the 13 minute mark, random frames occur for about 2-10 seconds. It happens to most of the minute marks after that. Going to run the same source through again without the MDegrain2 and see what happens.
Which decoder are you using? (LSmash or FFMS2)

[Atak_Snajpera]

Also what codec is used in source video?

[Ryushin]

Also what codec is used in source video?

From the m2ts source:
Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : 27
Duration : 3 h 17 min
Bit rate mode : Variable
Maximum bit rate : 38.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive


The second run through without MDegrain2 shows no random frames at least at first glance as I looked at a couple dozen minute mark locations where the randomness occurred before.

The decoder was LSmash.
#VideoSource
LoadPlugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\lsmash\LSMASHSource.dll")

I'm encoding to x265.

[Ryushin]

I looked at a 4K and HD encode I did with MDegrain2 and at first glance, it seems fine.

I typed this wrong. I did a 4K and HD (Godzilla: King of the Monsters) WITHOUT MDegrain and at first glance, it seems fine.

[Atak_Snajpera]

From the m2ts source:
Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : 27
Duration : 3 h 17 min
Bit rate mode : Variable
Maximum bit rate : 38.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive


The second run through without MDegrain2 shows no random frames at least at first glance as I looked at a couple dozen minute mark locations where the randomness occurred before.

The decoder was LSmash.
#VideoSource
LoadPlugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\lsmash\LSMASHSource.dll")

I'm encoding to x265.

Change default decoders like in this screenshot
https://i.imgsafe.org/91/91d45bdc49.png

[kempodragon]

I've finally found out why the encoding client couldn't find the index file it was looking for. First, I went back to v 1.24 to confirm my test files hadn't got corrupted since I had already encoded them before with it. Everything worked smoothly and that when I noticed the message "Indexing file, please wait" right after demuxing. I then was able to encode with no problems. Afterwards, I upgraded to 1.25.0, and retested. The message never appeared, only "Gathering information, please wait". Task manager showed ffprobe.exe running. When I tried to run the encoding client, the missing index file message appeared again. I then opened the job folder and spotted IndexStreams.cmd . Running it created the missing index file. For some reason, Ripbot is skipping creating an index file, even when the decoder is set to ffms.

I found another bug in the encoding client. In 1.24, I had 4 servers created in the settings tab and it activated and ran all of them with no problems. In the newest version, I had to manually activate and connect each server. On the plus side, Atak's code improvement actually dropped the cpu usage down to where I could activate a 5th server on my Threadripper. :D

For the record, my first test file is a game capture with UtVideo as the video codec,1080p60 and PCM as audio. My second file is from my camcorder, with 1080p60 and Dolby 5.1 audio.

[Ryushin]

Hi Atak,

So I've been assuming something and it came back to bit me and I'm wondering if it is by design or not.

So I make 720p versions of all my 4K/HD sources to store on mobile devices and for streaming. So I normally I just select the "HD Ready 1280x720" option creating the small version. The behavior I just noticed is that it will add borders to make it 16x9 when I have a video that is already cropped.

When I first started doing this, I don't remember it behaving in this way. I thought it was the 1280 or the 720 to scale the video to it's longest dimension. I.e. 2.4:1 - 1920x960 becomes 1280x640 and 4:3 - 1440x1080 becomes 960x720.

Now to get the behavior I want, I have to select Custom [X,XX:1]. It's easier If I select HD Ready first and then select Custom so I don't need to type in the dimension.

Can a option be added to not add borders when resizing video?

[Atak_Snajpera]

use Custom preset with aspect ratio in brackets (for example Custom [2,40:1])

[Ryushin]

use Custom preset with aspect ratio in brackets (for example Custom [2,40:1])

That will only work on sources that are exactly that aspect ratio. I was looking for a more general solution that I can just select HD Ready and it will scale the video appropriately without adding borders that were not there before.

[Pino72]

Change default decoders like in this screenshot
https://i.imgsafe.org/91/91d45bdc49.png

Setting FFMS2 option for HEVC solved all my problems I had when using LSMASH! Thanks Atak for making this available so easily.

[stryker412]

Atak, quick question. I purchased a BD of the old Scooby Doo series. I tried to make h265 encodes of the first disc. I normally don't do h265 because my computer is older and takes forever, but these are short episodes so I thought I'd try. I get the following error on each episode.

D:\>"D:\Tools\RipBot264v1.22.0\tools\mkvtoolnix\mkvmerge.exe" -o "D:\S01E02.mkv" --compression 0:none --title "S01E02" --default-duration 0:24000/1001fps "D:\Temp\RipBot264temp\video.265" --chapters "D:\Temp\RipBot264temp\job2\chapters.txt"
mkvmerge v35.0.0 ('All The Love In The World') 64-bit
Error: Unknown chapter file format in 'D:\Temp\RipBot264temp\job2\chapters.txt'. It does not contain a supported chapter format.

These are direct rips from the BD using MakeMKV. I've never once had an issue with that software. Any idea why RipBot doesn't like it? I'm on v1.25.

Edit: I just noticed too that there's no audio picked up in RipBot, even though the file has a stereo track. Could this be another 1.25 bug?

[Atak_Snajpera]

Have you checked that chapters.txt file?

[stryker412]

Yes

Error: (mkvextract) This file could not be opened or parsed.


Here is the Mediainfo for this file.

Format : Matroska
Format version : Version 2
File size : 1.67 GiB
Duration : 22 min 15 s
Overall bit rate mode : Variable
Overall bit rate : 10.8 Mb/s
Movie name : Scooby-Doo™, Where Are You! Disc 1
Encoded date : UTC 2019-09-06 22:15:49
Writing application : MakeMKV v1.14.4 win(x64-release)
Writing library : libmakemkv v1.14.4 (1.3.5/1.4.7) win(x64-release)

Video
ID : 1
ID in the original source medium : 4113 (0x1011)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, RefFrames : 4 frames
Format settings, GOP : M=1, N=12
Codec ID : V_MPEG4/ISO/AVC
Duration : 22 min 15 s
Bit rate mode : Variable
Bit rate : 10.6 Mb/s
Maximum bit rate : 11.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.212
Stream size : 1.64 GiB (98%)
Language : English
Default : No
Forced : No
Original source medium : Blu-ray

Audio
ID : 2
ID in the original source medium : 4352 (0x1100)
Format : AC-3
Format/Info : Audio Coding 3
Codec ID : A_AC3
Duration : 22 min 15 s
Bit rate mode : Constant
Bit rate : 192 kb/s
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Bit depth : 16 bits
Compression mode : Lossy
Stream size : 30.6 MiB (2%)
Title : Stereo
Language : English
Service kind : Complete Main
Default : Yes
Forced : No
Original source medium : Blu-ray

Menu
00:00:00.000 : en:Chapter 01
00:01:05.857 : en:Chapter 02
00:06:28.513 : en:Chapter 03
00:10:51.275 : en:Chapter 04
00:16:01.669 : en:Chapter 05
00:21:07.140 : en:Chapter 06

[slalom]

LSMASH did great today on mpeg2. No delays

[lemaireus]

Here is a copy of the error log I got:


E:\>"C:\Program Files (x86)\RipBot264\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "E:\Temp\RipBot264temp\job9\job9.avs" -strict -1 -f yuv4mpegpipe - | "C:\Program Files (x86)\RipBot264\tools\x265\x265_x64.exe" --colorprim bt709 --transfer bt709 --colormatrix bt709 --pass 1 --bitrate 2320 --stats "E:\Temp\RipBot264temp\job9\job9.stats" --fps 24000/1001 --min-keyint 24 --keyint 240 --frames 172056 --sar 1:1 --profile main10 --output-depth 10 --ctu 32 --no-slow-firstpass --y4m --output NUL -
y4m [info]: 1920x800 fps 24000/1001 i420p8 sar 1:1 unknown frame count
raw [info]: output file: NUL
x265 [info]: HEVC encoder version 3.1+14-f08461fdae33
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit+8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 2 / wpp(25 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : dia / 57 / 2 / 1
x265 [info]: Keyframe min / max / scenecut / bias: 24 / 240 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 1 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-2320 kbps / 0.60
x265 [info]: tools: rd=2 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: fast-intra strong-intra-smoothing lslices=5 deblock sao
x265 [info]: tools: stats-write

x265 [info]: frame I: 2080, Avg QP:20.15 kb/s: 14026.26
x265 [info]: frame P: 39753, Avg QP:21.44 kb/s: 4995.98
x265 [info]: frame B: 130223, Avg QP:25.09 kb/s: 1219.70
x265 [info]: Weighted P-Frames: Y:7.9% UV:6.7%
x265 [info]: consecutive B-frames: 9.2% 4.1% 6.6% 26.3% 53.8%

encoded 172056 frames in 18344.01s (9.38 fps), 2247.02 kb/s, Avg QP:24.18

E:\>"C:\Program Files (x86)\RipBot264\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "E:\Temp\RipBot264temp\job9\job9.avs" -strict -1 -f yuv4mpegpipe - | "C:\Program Files (x86)\RipBot264\tools\x265\x265_x64.exe" --colorprim bt709 --transfer bt709 --colormatrix bt709 --pass 2 --bitrate 2320 --stats "E:\Temp\RipBot264temp\job9\job9.stats" --fps 24000/1001 --min-keyint 24 --keyint 240 --frames 172056 --sar 1:1 --profile main10 --output-depth 10 --ctu 32 --y4m --output "E:\Temp\RipBot264temp\video.265" -
y4m [info]: 1920x800 fps 24000/1001 i420p8 sar 1:1 unknown frame count
raw [info]: output file: E:\Temp\RipBot264temp\video.265
x265 [info]: HEVC encoder version 3.1+14-f08461fdae33
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit+8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 2 / wpp(25 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 24 / 240 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-2320 kbps / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing lslices=5 deblock sao stats-read

x265 [info]: frame I: 2080, Avg QP:19.54 kb/s: 16468.27
x265 [info]: frame P: 39753, Avg QP:22.03 kb/s: 5039.21
x265 [info]: frame B: 130223, Avg QP:25.85 kb/s: 1261.56
x265 [info]: Weighted P-Frames: Y:4.3% UV:3.6%
x265 [info]: consecutive B-frames: 9.2% 4.1% 6.6% 26.3% 53.8%

encoded 172056 frames in 19715.15s (8.73 fps), 2318.21 kb/s, Avg QP:24.89

E:\>"C:\Program Files (x86)\RipBot264\tools\mkvtoolnix\mkvmerge.exe" -o "E:\New Rips\Ripbot\Olympus Has Fallen.mkv" --compression 0:none --title "Olympus Has Fallen" --default-duration 0:24000/1001fps "E:\Temp\RipBot264temp\video.265" --compression 0:none --language 0:eng "E:\Temp\RipBot264temp\job9\Encoded_Audio_1.ac3" --default-track 0:yes --forced-track 0:yes --language 0:eng -s 0 -D -A "E:\Temp\RipBot264temp\job9\6_subtitles_English_FORCED_1080.sup" --default-track 0:no --language 0:eng -s 0 -D -A "E:\Temp\RipBot264temp\job9\6_subtitles_English_1080.sup" --chapters "E:\Temp\RipBot264temp\job9\chapters.txt" --attachment-mime-type image/jpeg --attachment-description Cover --attachment-name Cover --attach-file "E:\Temp\RipBot264temp\job9\cover.jpg"
mkvmerge v37.0.0 ('Leave It') 64-bit
Error: The file 'E:\Temp\RipBot264temp\job9\Encoded_Audio_1.ac3' could not be opened for reading: open file error.
-------------------------

Elapsed Time: 10h:34m:44s

[lemaireus]

I have already copied the error log above, at the bottom of page 865.

I encode into x265, two passes, and always encode multichannel lossless master audio tracks into 640k AC3.

It's very frustrating getting that kind of an error at the end of 10:34 hours of work. This is the third file on which I am getting that error. Apparently the audio encoder does not encode DTS-MA track into AC3, which is why mkvmerge cannot find that track. Yes, I have checked the job folder each time, and there has never been an 'Encoded_Audio_1.ac3' file in the folder each time I get this error.

If Ripbot had not immediately deleted the encoded video, then I would have encoded and muxed the audio track myself in cases of such errors.

Atak, would you please consider that option: leave the encoded video intact in the job folder, and delete that only when all the temporary files are deleted? Such a simple option can save hours and hours of work going to waste in case of an error such as the one I posted above, and also in cases where the error is 'video.265 could not be opened.' (I used to get that sometimes in the past, not anymore).

Please leave the encoded video intact in the job folder for the file being encoded, for a user to delete as and when they please.

[slalom]

If Ripbot had not immediately deleted the encoded video, then I would have encoded and muxed the audio track myself in cases of such errors.
Run CombineAllChunks.cmd in chunks folder to get the video

[stryker412]

I posted my error log on the previous page too. Not sure why all the sudden I'm getting no audio and the chapters cannot be read.

[slalom]

Download the file from the first page and update that
That's what I do when I have trouble. Replace your files except .ini's

[GZZ]

Cosmetic Change. Can the text in the DE window be breaked into 2 lines or can the window be enlarged. As some of the text is truncated until I resize the column on every DE start.

https://imgur.com/YMFB3tb

[lemaireus]

Run CombineAllChunks.cmd in chunks folder to get the video

Thank you, Slalom. The question for a relative novice with cmd is 'how'?

May I assume that by 'chunks folder' you mean the job folder, thus, for example, 'job1'? How to I run 'CombineAllChunks.cmd' in that folder. A run down in three to four steps will be very helpful, and much appreciated.

Thanks a lot for what I hope will soon become a working suggestion.

[stryker412]

Atak, any suggestions?

https://forum.doom9.org/showthread.php?p=1884279#post1884279

[Atak_Snajpera]

Atak, any suggestions?

https://forum.doom9.org/showthread.php?p=1884279#post1884279

https://forum.doom9.org/showthread.php?p=1884278#post1884278

[lemaireus]

Run CombineAllChunks.cmd in chunks folder to get the video

Slalom, I did a bit of work on your suggestion, and there is an issue. I think you are working with the assumption that I am using distributed encoding. I am not. I am encoding my video on the one machine I have.

Atak, would you please consider the option of leaving the encoded video intact in the job folder, and delete that only when all the temporary files are deliberately deleted by a user? Such a simple option can save hours and hours of work from going to waste in case of an audio or video muxing error (Encoded_Audio_1.ac3 cannot be opened, or video.265 cannot be opened).

Here are references back to my previous two posts on the issue, with the first showing the error log:
https://forum.doom9.org/showthread.php?p=1884332#post1884332
https://forum.doom9.org/showthread.php?p=1884333#post1884333

[slalom]

Slalom, I did a bit of work on your suggestion, and there is an issue. I think you are working with the assumption that I am using distributed encoding. I am not. I am encoding my video on the one machine I have.
Actually, yes I was :o

[stryker412]

https://forum.doom9.org/showthread.php?p=1884278#post1884278


Yes the contents of that file were in my post.

[GZZ]

I see FFMS2 Video decoder is default. What is the difference and benefits from one another FFMS2 vs. LSMASH ?

[lemaireus]

This is becoming routine, and therefore very, very frustrating. Got the error once again today; each time I have to encode DTS/DTS-MA audio into AC3, at the end of a two-pass encode I get the message:

Error: The file 'E:\Temp\RipBot264temp\job2\Encoded_Audio_1.ac3' could not be opened for reading: open file error.

I can copy the entire log file, if you like.

Atak, the solution I have suggested is really a straightforward one, even if it takes time writing some fresh code, but it will save a lot of people a lot of frustration if they could simply pick up the encoded video and mux is manually in case of these errors, rather than having to do the entire encode from scratch.

Please allow Ripbot to leave the encoded video in the job folder, till a user deletes that job folder manually. That is all it will take to save hours and hours of encoding from being flushed down the drain.

Keeping my fingers crossed for a positive response from you.

[byteshare]

I see FFMS2 Video decoder is default. What is the difference and benefits from one another FFMS2 vs. LSMASH ?
LSMASH is the newer one added to RipBot, so if you never had any issues with FFMS2 I'd use that, but LSMASH works differently so that if you have issues with frame accuracy on a source such as MPEG2 then use that one.

This is becoming routine, and therefore very, very frustrating. Got the error once again today; each time I have to encode DTS/DTS-MA audio into AC3, at the end of a two-pass encode I get the message:

Error: The file 'E:\Temp\RipBot264temp\job2\Encoded_Audio_1.ac3' could not be opened for reading: open file error.

I can copy the entire log file, if you like.

Atak, the solution I have suggested is really a straightforward one, even if it takes time writing some fresh code, but it will save a lot of people a lot of frustration if they could simply pick up the encoded video and mux is manually in case of these errors, rather than having to do the entire encode from scratch.

Please allow Ripbot to leave the encoded video in the job folder, till a user deletes that job folder manually. That is all it will take to save hours and hours of encoding from being flushed down the drain.

Keeping my fingers crossed for a positive response from you.
Have you posted a sample of the source here?
Does it happen consistently on the same source, aka if you ran a sample that had an issue again and again does it happen each time?

[lemaireus]

LSMASH is the newer one added to RipBot, so if you never had any issues with FFMS2 I'd use that, but LSMASH works differently so that if you have issues with frame accuracy on a source such as MPEG2 then use that one.

Have you posted a sample of the source here?
Does it happen consistently on the same source, aka if you ran a sample that had an issue again and again does it happen each time?

The fist section of the reply was to GZZ, but in my case, I never had problems of the kind I do before the move to LSMASH. Frame accuracy was never an issue with me. My sources are usually BD/BDRemuxes and never did I ever have a problem with FFMS2 in any audio conversions.

The issue has been coming up since the move to LSMASH. I have had the error thrice since that shift, with three difference sources. Common to these three sources was a DTS audio track: lossless DTS-MA tracks in two sources, and 1504k DTS track in the third source. I haven't yet tried converting .thd audio. Will give that a run through and post an update.

I shall do a thorough test tomorrow of samples with three different kinds of audios: DTS-MA, DTS, and TrueHD. Shall post the results after that, and will upload a sample in case that is necessary for analysis.

More tomorrow. Desperately looking for a solution to this because the problem is intensely frustrating: it ruins the encode right at the end of two passes, when it's time to mux the audio-video streams.

[stryker412]

Ripbot (newest version) is not picking up the audio track or the chapter files in my latest TV show rip. The tracks are indeed there as listed in MediaInfo and using VLC. Anyone else seeing this?

[GZZ]

The fist section of the reply was to GZZ, but in my case, I never had problems of the kind I do before the move to LSMASH. Frame accuracy was never an issue with me. My sources are usually BD/BDRemuxes and never did I ever have a problem with FFMS2 in any audio conversions.


Thanks for the explanation, though there was some extra decoding speed hidden in LSMASH, but if it just an alternative to those who have frame accuracy issue, then I leave it a FFMS2 as it has worked fine so far.

[sneaker_ger]

You will always find samples that make problems in L-SMASH but work fine in ffms2 and vice versa. You can't win.

[duffbeer]

99.9% of the time FFMS2 works perfectly. LSMASH has caused massive problems.

[Atak_Snajpera]

99.9% of the time FFMS2 works perfectly. LSMASH has caused massive problems.

Ffms2 is more broken than i thought. Latest version of lsmash
(Not included in ripbot) fixed vc1 decoding and that random skips in avc and hevc. Just test your files in my tool seektester to see what i mean.

[lemaireus]

Update regarding the following posts:

https://forum.doom9.org/showthread.php?p=1884332#post1884332
https://forum.doom9.org/showthread.php?p=1884333#post1884333
https://forum.doom9.org/showthread.php?p=1884690#post1884690
https://forum.doom9.org/showthread.php?p=1884700#post1884700

I ran the tests and here are the results: every encode that involved audio encoding on a setting other than 'copy stream' failed. None of the job folders of the encodes which gave the error contained the new encoded audio file which should have been there. The inference was obvious: that EAC3 wasn't kicking in at the start of the encoding process because, most likely, the file with the command line code meant to start EAC3 might have become corrupted.

Once the problem was diagnosed, it was the small matter of testing a solution. I downloaded a fresh copy of Ripbot, updated it, and ran the test encodes again, and everything is now working fine.

Essentially, I went through all the frustration I did because of the corruption of the file with the command line that was meant to start EAC3 for encoding audio, and which stopped working at some stage.

Thank you very much for your suggestions, and I'll keep my fingers crossed that the encodes proceed smoothly hereafter.

[byteshare]

Ffms2 is more broken than i thought. Latest version of lsmash
(Not included in ripbot) fixed vc1 decoding and that random skips in avc and hevc. Just test your files in my tool seektester to see what i mean.
I'd be willing to try LSmash again once the new version is added to RB, if it fixed the seek/frame issues that were happening before.
How is FFMS2 broken? I'm not seeing issues with it at the moment.

[Atak_Snajpera]

I'd be willing to try LSmash again once the new version is added to RB, if it fixed the seek/frame issues that were happening before.
How is FFMS2 broken? I'm not seeing issues with it at the moment.

https://forum.doom9.org/showthread.php?p=1884507#post1884507

[stryker412]

Tried a number of different things such as older versions of RipBot, manually updating eac3to/mkvtoolnix/etc. Nothing has worked. Been using RipBot for years and now it won't load any audio tracks from this rip. Of all things... It's the recent Scooby Doo BD release. The audio tracks are there (they play fine in VLC). At a loss here.

[cypher007]

hello, awesome program been using it for years.

but I have one feature request. if I have an MKV with a DTS-MA sound track I would like to be able to recode it with just DTS core. I can do this with a pure blu ray using ripbot as it will let me select core. but it seems I have no options like this when dealing with MKV's.

[slalom]

Use tsMuxer to get the core

Once the problem was diagnosed, it was the small matter of testing a solution. I downloaded a fresh copy of Ripbot, updated it, and ran the test encodes again, and everything is now working fine.

Essentially, I went through all the frustration I did because of the corruption of the file with the command line that was meant to start EAC3 for encoding audio, and which stopped working at some stage.

Thank you very much for your suggestions, and I'll keep my fingers crossed that the encodes proceed smoothly hereafter.
Told you! :D

[Viper714]

So, I just finished encoding a 4K movie, The Mummy 3" and it took 6 hours and 40 minutes to encode in x265. Should I do 264 instead. That is way to looong with four machines running in DE. I can normally do 1080 blu rays in x264 in about 45-60 minutes. I just encode video and do a copy stream on audio. What is the best setting for encoding 4K movies? Any suggestions are helpful...

Below is a sample of my conversion times. Disregard the Aquaman times it was a test only.
https://i.imgur.com/DpV4AFR.png

[Atak_Snajpera]

There is no best settings ... 4x pixels to encode plus x265 Has to take a long time.

[stryker412]

Atak, I downloaded Staxrip and was able to load up the source files without problem. Any idea why Ripbot cannot pull up the audio or chapters for these?

https://forum.doom9.org/showpost.php?p=1884279&postcount=17298
https://forum.doom9.org/showpost.php?p=1884823&postcount=17324

[Atak_Snajpera]

Maybe filename Has some non standard (unicode)characters.

[byteshare]

Maybe filename Has some non standard (unicode)characters.
Yeah, I setup a regex batch (antRenamer) to strip those from my files before I add them to RipBot.

[emmaurilio]

Hi all!

I´m having some problems with encoding with RipBot1.21 and above. I tried to find a topic about this problem without success. The solution may be quite simple, but I don't know a lot about the details of the encoder plugins. That´s why I ask your help.

The files bellow will show better what is happening, but in short the final file had a green frame added after each normal frame of the original video.

https://www.dropbox.com/sh/a2eu6dlanqr2plh/AAD3TpZg-Xls2EkTlwcFVxY_a?dl=0

The encoding is working well until version 1.20. I suspect that the problem is occurring with the AviSynth.

Can someone help me?

[Atak_Snajpera]

Hi all!

I´m having some problems with encoding with RipBot1.21 and above. I tried to find a topic about this problem without success. The solution may be quite simple, but I don't know a lot about the details of the encoder plugins. That´s why I ask your help.

The files bellow will show better what is happening, but in short the final file had a green frame added after each normal frame of the original video.

https://www.dropbox.com/sh/a2eu6dlanqr2plh/AAD3TpZg-Xls2EkTlwcFVxY_a?dl=0

The encoding is working well until version 1.20. I suspect that the problem is occurring with the AviSynth.

Can someone help me?

Will be fixed in next auto update. Be patient.

Temporary hack -> rename .mpg to .ts

[emmaurilio]

Will be fixed in next auto update. Be patient.

Temporary hack -> rename .mpg to .ts


Thanks for the answer!

I'm patient :) , but just to understand how it works... is there a prediction or even vague idea of the timing of this next update? I am using version 1.20, so I disabled autoupdate and because that I will not know when it occurs. :/

Thanks.

[emmaurilio]

Will be fixed in next auto update. Be patient.

Temporary hack -> rename .mpg to .ts


It works!!! Thanks!

[POTI]

I have an odd one. It just started happening yesterday. I am running win 10 all up to date. I started getting this error.
AVC Import results: 159138 samples (481657 NALUs) - Slices: 1460 I 53052 P 104626 B - 160553 SEI - 1414 IDR
Stream uses forward prediction - stream CTS offset: 2 frames
Opening file C:\Temp\RipBot264temp\job1\Encoded_Audio_1.aac failed
Error importing C:\Temp\RipBot264temp\job1\Encoded_Audio_1.aac:lang=und: Requested URL is not valid or cannot be found

I checked to make sure that the temp folder is shared and it is. The Encoded_Audio_1.aac is missing. I can go in and play the job and it all appears and sounds ok. Anything else you need from me to look into this? Happening on multiple encodes.

[ReinerSchweinlin]

I noticed machines running WIn10 slow down encoding when headless left alone. As soon as I log into them with Remote Desktop, the pace steps up. If I manually set the priority of ffmpeg/x265 to "above normal" - I get full usage of CPU - until the next chunk and it all slows down again.
Win7 machines in my swarm don't show this behaviour.
The win10 Machines are already cleaned from most background processes.

I tried setting "priority above normal" in the server settings, but this doesn´t seem to change anything.

Any advice?

[byteshare]

I noticed machines running WIn10 slow down encoding when headless left alone. As soon as I log into them with Remote Desktop, the pace steps up. If I manually set the priority of ffmpeg/x265 to "above normal" - I get full usage of CPU - until the next chunk and it all slows down again.
Win7 machines in my swarm don't show this behaviour.
The win10 Machines are already cleaned from most background processes.

I tried setting "priority above normal" in the server settings, but this doesn´t seem to change anything.

Any advice?
Leave them logged in? I just leave mine logged in (not locked) and don't see an issue with the CPU like that.

[Atak_Snajpera]

I noticed machines running WIn10 slow down encoding when headless left alone. As soon as I log into them with Remote Desktop, the pace steps up. If I manually set the priority of ffmpeg/x265 to "above normal" - I get full usage of CPU - until the next chunk and it all slows down again.
Win7 machines in my swarm don't show this behaviour.
The win10 Machines are already cleaned from most background processes.

I tried setting "priority above normal" in the server settings, but this doesn´t seem to change anything.

Any advice?

Maybe it is some power saving strategy in win10? I can't help here because I'm using only win7.
https://www.thewindowsclub.com/power-throttling-windows-10

[ReinerSchweinlin]

Thanx to both of you for replying.
@byteshare
I can´t leave all of them logged in, so this sadly is no option for me.
@Atak
Would it be possible to implement "priority above normal" for the startup of the servers`?

[Atak_Snajpera]

Just use Process Hacker and save process priority for ffmpeg.exe and x264.exe/x265.exe
https://i.imgsafe.org/9f/9f77c38193.png

[silentstrife]

Hello,

I am a long time user/reader, first time poster. Great program, have been using it for a very long time. I just got the recent update and after processing a movie, I have no video, just a cover picture, and audio only. I have tried using both of the 2 new encoders and set them across the board for the options. Has any one had this happen or reported yet? I haven't seen anything in the last few pages.

Also, when i tried to use DE, it failed on all of my video conversion attempts. Unfortunately i didn't screen shot or see any error logs to provide.

Error Log:
D:\>"D:\Blu Ray Tools\RipBot264v1.24.0\EncodingClient.exe" "D:\Temp\RipBot264temp\job1\job1_EncodingClient.meta"

D:\>"D:\Blu Ray Tools\RipBot264v1.24.0\tools\mkvtoolnix\mkvmerge.exe" -o "D:\video.mkv" --compression 0:none --title "video" --default-duration 0:24000/1001fps "D:\Temp\RipBot264temp\video.265" --compression 0:none --language 0:eng --aac-is-sbr 0:0 "D:\Temp\RipBot264temp\job1\Encoded_Audio_1.aac" --chapters "D:\Temp\RipBot264temp\job1\chapters.txt"
mkvmerge v37.0.0 ('Leave It') 64-bit
Error: The file 'D:\Temp\RipBot264temp\video.265' could not be opened for reading: open file error.
-------------------------

Elapsed Time: 02h:58m:03s

Manually combine the chuncks, ran Job1_MuxFiles.bat, found output video at expected location, opened to find no video at all but has audio.

RB Ssettings:
MPEG-2 and Other set to LSMASH, the others are set to FFMS2.

[Atak_Snajpera]

Try again with new fresh job .

[ReinerSchweinlin]

Hey Atak,

thank you for the suggestion, didn´t know that process hacker can do that. A first quick try didn´t help though. Still dropping CPU Usage when left alone.. For some reason I cannot change the power scheme permanently, Windows always reverts it after a few hours. Seems to be a standard behaviour, since I discovered this on several WIN10 machines which are completely different (win64/32/home/pro/etc, different CPUs..). I will have to do more investigation on this.

[Atak_Snajpera]

I hope you left process hacker running in background because otherwise those processes will run with standard normal priority.

[silentstrife]

Try again with new fresh job .

Hi Atk,

Yes that was a fresh job. The only thing i know happened was the update from version 1.24 to 1.25. I might just go back to 1.24 and disable the auto-update. If you can think of any info i can provide to help fix this issue please let me know.

RB Job Settings - Profile - MA10 X.X 10 bit progressive HEVC, CQ 20, Denoise - MDegrain2 + KNLMeansCL, audio 2.0

Basic System info:
Windows 10 Pro V.1903
CPU - i7-5960x
RAM - 32GB
GFX - RTX 2070

[LigH]

@ReinerSchweinlin: Microsoft SysInternals' "ProcessExplorer" can do that too. It is my default Task Manager now.

[Atak_Snajpera]

Hi Atk,

Yes that was a fresh job. The only thing i know happened was the update from version 1.24 to 1.25. I might just go back to 1.24 and disable the auto-update. If you can think of any info i can provide to help fix this issue please let me know.

RB Job Settings - Profile - MA10 X.X 10 bit progressive HEVC, CQ 20, Denoise - MDegrain2 + KNLMeansCL, audio 2.0

Basic System info:
Windows 10 Pro V.1903
CPU - i7-5960x
RAM - 32GB
GFX - RTX 2070

Compress whole job folder (without big files like video.mkv or audio files) and send IT to me. I will take a look...

[ReinerSchweinlin]

I hope you left process hacker running in background because otherwise those processes will run with standard normal priority.

Of course
@LigH
Thanx for the info.

[Atak_Snajpera]

Of course
@LigH
Thanx for the info.

Have you verified that process in fact gets higher priority?

[silentstrife]

Compress whole job folder (without big files like video.mkv or audio files) and send IT to me. I will take a look...

Sorry to have troubled you. I re-installed my codec pack and that seems to have fixed what ever was causing my issue.

Have a good day.

[byteshare]

New x265 3.2
https://bitbucket.org/multicoreware/x265/raw/353572437201d551381002aebf20d244bd49ef17/doc/reST/releasenotes.rst
Version 3.2
===========

Release date - 25th September, 2019.

New features
------------
1. 3-level hierarchical motion estimation using :option:`--hme` and :option:`--hme-search`.
2. New AQ mode (:option:`--aq-mode` 4) with variance and edge information.
3. :option:`selective-sao` to selectively enable SAO at slice level.

Enhancements to existing features
---------------------------------
1. New implementation of :option:`--refine-mv` with 3 refinement levels.

Encoder enhancements
--------------------
1. Improved quality in the frames following dark scenes in ABR mode.

API changes
-----------
1. Additions to x265_param structure to support the newly added features :option:`--hme`, :option:`--hme-search` and :option:`selective-sao`.

Bug fixes
---------
1. Fixed encoder crash with :option:`--zonefile` during failures in encoder_open().
2. Fixed JSON11 build errors with HDR10+ on MacOS high sierra.
3. Signalling out of range scaling list data fixed.
4. Inconsistent output fix for 2-pass rate-control with cutree ON.

Known issues
------------
1. Build dependency on changeset cf37911 of SVT-HEVC.
Update: Thank you for updating RB!

[byteshare]

I have been having some slowness issues when doing any filters with my GPU, such as KNLmeans.
Looking at Process Explorer a bit more I realized that FFMpeg didn't show any GPU usage (GPU, GPU System Bytes, GPU Dedicated Bytes, GPU Committed Bytes)
If I run the RipBot script in AVSPmod I do see GPU System Bytes, GPU Dedicated Bytes, and GPU Committed Bytes showing usage.
The AVS Meter also behaves strangely, where the text fields go over the line and I have to scroll down to see all of the data.
The script runs in RipBot (just using the Adaptive KNLMeansCL in RipBot to test with) and a job will complete. The output looks like I expect just very slow to get it.
When running a job using KNLmeans I can't get a lot of CPU usage because my guess is the decoder (FFMpeg) can't supply enough to process). If I use a non-GPU filter the CPU can basically max out. Using non-DE mode at the moment.
Any ideas what is happening?

[Atak_Snajpera]

I have been having some slowness issues when doing any filters with my GPU, such as KNLmeans.
Looking at Process Explorer a bit more I realized that FFMpeg didn't show any GPU usage (GPU, GPU System Bytes, GPU Dedicated Bytes, GPU Committed Bytes)
If I run the RipBot script in AVSPmod I do see GPU System Bytes, GPU Dedicated Bytes, and GPU Committed Bytes showing usage.
The AVS Meter also behaves strangely, where the text fields go over the line and I have to scroll down to see all of the data.
The script runs in RipBot (just using the Adaptive KNLMeansCL in RipBot to test with) and a job will complete. The output looks like I expect just very slow to get it.
When running a job using KNLmeans I can't get a lot of CPU usage because my guess is the decoder (FFMpeg) can't supply enough to process). If I use a non-GPU filter the CPU can basically max out. Using non-DE mode at the moment.
Any ideas what is happening?
Run Process Explorer as Administrator to see GPU usage on ffmpeg.exe. You most likely have bottleneck on GPU. KNLMeansCL is extremely intensive even for most modern GPUs.

[byteshare]

Run Process Explorer as Administrator to see GPU usage on ffmpeg.exe. You most likely have bottleneck on GPU. KNLMeansCL is extremely intensive even for most modern GPUs.
Running it as Admin does show the bytes used.
Hmmm...now to figure out why I'm getting ~1-2FPS with a Geforce 980 and Ryzen 1700. I get without the GPU filter ~20FPS with the same x265 settings. I'm using about 1.7GB of GPU memory (of 4GB) from FFMpeg but the GPU core load (using Open Hardware Monitor) bounces between 0-13%.
I've been working "around" it by using other encoders to use more CPU on sources that don't need filters. I would use DE mode but the second server usually never starts.
I don't have an older version of RB to try, I think 1.23 but up to some point in 1.24 before something was changed. Is the one linked on the first page the first version of 1.24? I recall getting 8-12FPS with the same settings, granted different sources, but getting 1-2FPS on several sources with the most recent version of RBk./.
I'll try updating my drivers for now. update: didn't seem to make a different. I also changed the following to no avail: my power profile to high performance and same for the Nvidia Control Panel "Power management mode" to "Perfer maximum performance".

[Atak_Snajpera]

How many FPS do you get in avsmeter with only knlmeans enabled? 2fps on 1920x1080 is indeed very slow. I also get 2fps but on GeForce 710 (99% gpu usage)

[byteshare]

How many FPS do you get in avsmeter with only knlmeans enabled? 2fps on 1920x1080 is indeed very slow. I also get 2fps but on GeForce 710 (99% gpu usage)

With AVSMeter
KNLmeansCL h=2 27FPS 90-95% GPU
KNLmeansCL Adaptive h=2 ??FPS 80-90% GPU (AVSMeter shows the information scrolling so I can't see true FPS)
KNLmeansCL Adaptive + MD2 h=2 ??FPS 0-20% GPU (AVSMeter shows the information scrolling so I can't see true FPS)

In the OpenCL setup, I have x264 OpenCL acceleration set to no (none) device and KNLMeansCL set to Geforce GTX 980
Turning that on for x264 doesn't seem to help, but it does seem worse getting ~0.7 FPS.

Running same HEVC (not with AVSmeter) with just these filters: `FastLineDarkenMOD4().LSFmod()` (video is in the actual script, so I didn't forget that :D ) I'm getting ~12 FPS

[Atak_Snajpera]

Mdegrain2 running on CPU is a real bottleneck in your case. How many FPS do you get with mdegrain2 only?

[duffbeer]

I'm having an issue encoding audio to AC3 at the moment.
For some reason, RipBot produces an error any time I re-encode the audio to AC3. The error report states that the encoded audio file cannot be found.
My sources are either full disc structure or MKV. Sometimes with DTS audio and sometimes Dolby Atmos.
It always used to work perfectly so I'm not sure what has changed or whether it may be a local problem for me. Any ideas Atak?

[Atak_Snajpera]

I'm having an issue encoding audio to AC3 at the moment.
For some reason, RipBot produces an error any time I re-encode the audio to AC3. The error report states that the encoded audio file cannot be found.
My sources are either full disc structure or MKV. Sometimes with DTS audio and sometimes Dolby Atmos.
It always used to work perfectly so I'm not sure what has changed or whether it may be a local problem for me. Any ideas Atak?

You may try running manually jobx_EncodeAudioAll.cmd and then see what happens. I recommend doing this manually via console (cmd.exe) instead of just clicking on .cmd file.

[byteshare]

Mdegrain2 running on CPU is a real bottleneck in your case. How many FPS do you get with mdegrain2 only?
Can't get Mdegrain2 to work with AVSMeter, having the scrolling issue for some reason.
Had to run a job, used the same job that was getting ~12 FPS with these filters: `FastLineDarkenMOD4().LSFmod()`
Found I was only getting ~2FPS, so yeah, Mdegrain is the slow down (I was wrong, wasn't to do with my GPU, sorry), but (not using DEmode) only using ~33% (20%-40% seems to be the range) of the CPU during encode, so it seems the multiple processing threads isn't working for some reason? I have the "Limit to following filters only" unchecked.

[Atak_Snajpera]

Show me your full script

[byteshare]

Show me your full script

I ran it on a Ryzen 1700. I ran a similar test on a Ryzen 2700x and it is using more CPU ~60% in AVSmeter, which is showing correctly (not scrolling the data).
Update: Re-ran the above test after another reboot and an update to RB the AVSmeter isn't scrolling and not sure what you changed but it is using way more CPU and I'm getting ~9 FPS now!
Awesome, thank you!

[byteshare]

Something I'm noticing with the new update (now that my jobs are going faster!) is most of the jobs are failing at the remux step.
I don't think it is the filename, unless it is the "-" and if I manually run the batch file to make the video.265 it works fine and the video looks good.
I have at least 4 others I am able to see this happening on and two others that worked correctly. I am only encoding the video (skipping the demux step in the options to skip audio)

Update: Set my last update time to 2016, don't see any new updates, my Core is 2019.09.30.
Added a job that was failing again, and it failed again. only characters not ABC123, were brackets and a dash: "[" "]" "-"

[duffbeer]

How does RipBot handle HDR10+? Does it get passed through to the encoded MKV file or is it converted to HDR10?

[Atak_Snajpera]

No. You have to pass that data on your own.

[cypher007]

Noticed when encoding some anime, using very slow and animation tunes, that my intel i5 9100f doesn’t use 100% cpu. It seems to max 2 cores and about 70-80% the other 4 cores.

The result is 2-4fps. But if I open taskmgr it will then creep upto 16fps, this is on the first pass of a 2 pass encode.

So when I left some encodes to run 2 took nearly 3 hrs for 23min of video. But then the last 2 took 45mins. As I think I checked the cpu usage.

It’s like rip bot is dozing until the pc is used.

[Atak_Snajpera]

Most likely win10 is doing something "Smart". Change power profile to high performance

[userx]

Hello
Is there any possibility to automatically delete source file after successful end of job?

[Atak_Snajpera]

Hello
Is there any possibility to automatically delete source file after successful end of job?

Not yet. Maybe in next update.

[slalom]

There is a problem

If I run a job again, the program doesn't overwrite the previous file. I have to rename the old, for the new to be created

EDIT

To be more precise, running CombineAllChunks.cmd, doesn't overwrite existing video.264/5 in jobx folder

EDIT 2

Delete previous line

Something else is happening and overwrite doesn't work

[Ryushin]

Just wanted to bring this up again. I don't know how it would just be me though. I installed fresh versions of Ripbot264 a month or so ago and the 4K seamless branching is still causing issues. From a previous post:

I rip a 4K disc to the hard drive that has seamless branching, such as Incredibles 2. I then use Ripbot to pull in the movie and process it. The output will have out of sync audio towards the end of the movie. I think this only happens on 4K discs that are towards 2 hours or longer. Just had to go through this with Ralph Breaks the Internet and Snow White and the Huntsman Extended edition.

The work around I've been doing is to use MakeMKV to create a video file from the same source and then pull that into Ripbot to process it and it comes out fine.

This time the 4K disc in question is Ratatouille.

Is anyone else experiencing this?

[byteshare]

Just wanted to bring this up again. I don't know how it would just be me though. I installed fresh versions of Ripbot264 a month or so ago and the 4K seamless branching is still causing issues. From a previous post:

I rip a 4K disc to the hard drive that has seamless branching, such as Incredibles 2. I then use Ripbot to pull in the movie and process it. The output will have out of sync audio towards the end of the movie. I think this only happens on 4K discs that are towards 2 hours or longer. Just had to go through this with Ralph Breaks the Internet and Snow White and the Huntsman Extended edition.

The work around I've been doing is to use MakeMKV to create a video file from the same source and then pull that into Ripbot to process it and it comes out fine.

This time the 4K disc in question is Ratatouille.

Is anyone else experiencing this?
Which decoder are you using? And what is your Core version?

[Ryushin]

Which decoder are you using? And what is your Core version?

FFMS2 for the decoder and v1.25. Under tools, Core says 2019.09.01.

[byteshare]

FFMS2 for the decoder and v1.25. Under tools, Core says 2019.09.01.
You're using an old version, should be 2019.09.30 for the core

[cypher007]

Most likely win10 is doing something "Smart". Change power profile to high performance

I’ll try putting it on high performance and see what happens.

[GZZ]

Just wanted to bring this up again. I don't know how it would just be me though. I installed fresh versions of Ripbot264 a month or so ago and the 4K seamless branching is still causing issues. From a previous post:

I rip a 4K disc to the hard drive that has seamless branching, such as Incredibles 2. I then use Ripbot to pull in the movie and process it. The output will have out of sync audio towards the end of the movie. I think this only happens on 4K discs that are towards 2 hours or longer. Just had to go through this with Ralph Breaks the Internet and Snow White and the Huntsman Extended edition.

The work around I've been doing is to use MakeMKV to create a video file from the same source and then pull that into Ripbot to process it and it comes out fine.

This time the 4K disc in question is Ratatouille.

Is anyone else experiencing this?

Be carefull with MakeMKV, it has issues with Disney/Pixar Animation. I think its because of the seamless branching. I had alot of issue where ripbot will reject it saying it didnt have any audio. It turned out that MakeMKV does a poor job and there was a glitch in the movie and audio. I then switched to MKVToolNix to rip the main movie and it seem to do a proper job and havent got any AV sync issues.

[Ryushin]

You're using an old version, should be 2019.09.30 for the core
Updated to the latest version and it is still doing the same thing.


Be carefull with MakeMKV, it has issues with Disney/Pixar Animation. I think its because of the seamless branching. I had alot of issue where ripbot will reject it saying it didnt have any audio. It turned out that MakeMKV does a poor job and there was a glitch in the movie and audio. I then switched to MKVToolNix to rip the main movie and it seem to do a proper job and havent got any AV sync issues.
I'll keep an eye out for that. I'm using the MakeMKV for Linux and at least for me, it is making mkv files that are synced correctly and I import those into RipBot and I get the good output that is synced.

This only seems to occur on 4K content. Blu-ray seamless branching seems to work great and I have no issues.

[GZZ]

This only seems to occur on 4K content. Blu-ray seamless branching seems to work great and I have no issues.

I only had issue with Disney/Pixar and MakeMKV, I use it for all other movies without issue. So thinking its the way Disney/Pixar 4k UHD is created.

[byteshare]

There is a problem

If I run a job again, the program doesn't overwrite the previous file. I have to rename the old, for the new to be created

EDIT

To be more precise, running CombineAllChunks.cmd, doesn't overwrite existing video.264/5 in jobx folder

EDIT 2

Delete previous line

Something else is happening and overwrite doesn't work
For me it seems if I open RB, start jobs, abort, and then without restarting RB start the jobs again at the end of a job it won't mux.
--Update, might be more to it but I'm having the issue on more than one machine for jobs not muxing at the end (was only using one for a bit), so two separate machines with different sources running DE mode locally.
Running The batch CombineAllChunks then job1_MuxFiles works fine when I do it manually, so for now I'm run the combineallchunks then the jobxx_muxfiles batch manually. :/
"Z:\Temp\RipBot264temp\job1\video.265"
mkvmerge v37.0.0 ('Leave It') 64-bit
Error: The file 'Z:\Temp\RipBot264temp\job1\video.265' could not be opened for reading: open file error.

Side thing: I see .lwi files being made even though I have it set to FFMS2, is that intentional and if so why?

[Atak_Snajpera]

.lwi file is always created in order to avoid huge delay when you decide to switch from ffms2 to lsmash (indexing may take few minutes)

[byteshare]

.lwi file is always created in order to avoid huge delay when you decide to switch from ffms2 to lsmash (indexing may take few minutes)
Oh, didn't realize that was happening, but I also didn't realize that each job was getting muxed twice until with a more recent version it started breaking on me....
The more you know :D
Thank you for answering.
--To be clear the muxing issue isn't all of the time. If I can get it to work once then the following jobs keep working. Right now I've been restarting a job until it doesn't fail and then the jobs after work...then I don't touch anything because even adding more jobs seems to have a possibility of breaking it again.

[slalom]

For me it seems if I open RB, start jobs, abort, and then without restarting RB start the jobs again at the end of a job it won't mux.
--Update, might be more to it but I'm having the issue on more than one machine for jobs not muxing at the end (was only using one for a bit), so two separate machines with different sources running DE mode locally.
Running The batch CombineAllChunks then job1_MuxFiles works fine when I do it manually, so for now I'm run the combineallchunks then the jobxx_muxfiles batch manually. :/
"Z:\Temp\RipBot264temp\job1\video.265"
mkvmerge v37.0.0 ('Leave It') 64-bit
Error: The file 'Z:\Temp\RipBot264temp\job1\video.265' could not be opened for reading: open file error.

Side thing: I see .lwi files being made even though I have it set to FFMS2, is that intentional and if so why?
When the DE finishes, the next job starts. At least the files are there to do it manually

[userx]

Can report same behavor. CombineAllChunks.cmd has to be started manually. D:\Temp\RipBot264temp\job8\video.264 is not created before job8_MuxFiles.cmd is executed.
Very annoying. Some jobs succeeded and after any point all jobs finish with ERROR and no output files is created.

[mparade]

Your program is what I have been looking for in the past…
thank you very much!

In the past I used DGDecNV for video decoding, which is the preferred (most reliable if I can say something like this) one in Ripbot?

I think sometimes I would like to use some denoiser on my sources (very rarely when bitrates are getting higher very much). Do you have some guidelines when to use which?
I am reencoding only high bitrate HD sources but sometimes grain, noise doesn't let me reduce the bitrate in the amount I would like to.

Distributed encoding is just amazing on my 32 cores! I have never seen anything like this in other programs.

Thank you very much for your amazing program.

[Atak_Snajpera]

In the past I used DGDecNV for video decoding, which is the preferred (most reliable if I can say something like this) one in Ripbot?
LSMASH

I think sometimes I would like to use some denoiser on my sources (very rarely when bitrates are getting higher very much). Do you have some guidelines when to use which?
Just activate MDegrain2

[byteshare]

If anyone missed it. New update to x265 (3.2+5), MKVtoolnix (38.0.0), and Core (2019.10.06)
Thank you for the update.

How do we use the "Run Script after finished job"?

[Atak_Snajpera]

How do we use the "Run Script after finished job"?
What do you mean?

[byteshare]

What do you mean?
I don't see an option/where to define what script it will run, unless it is an internal script?
I can check it but I don't see what it does, so I don't know how to use it.
--Oh, thank you for adding GPU as a option in the settings for decoders and not just a right click option. Other than needing Intel or NVidia to use, any downsides to it over FFMS2 or LSMASH?

[Atak_Snajpera]

I don't see an option/where to define what script it will run, unless it is an internal script?
Just check Job finished successfully.cmd and Job finished with error.cmd

[byteshare]

Just check Job finished successfully.cmd and Job finished with error.cmd
Ah, thank you!

[Dhry]

I'm also encountering the video.265 could not be opened for reading: open file error issue. Started after a recent update, everything worked perfectly fine for months before, through multiple previous updates. And I too discovered that running the CombineAllChunks then then jobx_MuxFiles scripts manually will complete the process. Please fix.

Also, separate question: Are there any plans for a Linux encodingserver? I have a couple of Linux machines that I'd like to throw into the farm.

Thanks for the software, it's fantastic!
Dhry

[Atak_Snajpera]

Install windows (win10 works without activation forever) on virtual machine and you are good to go.

[mparade]

should I ignore it?:

x264 [warning]: OpenCL: not compiled with OpenCL support, disabling

[byteshare]

I'm also encountering the video.265 could not be opened for reading: open file error issue. Started after a recent update, everything worked perfectly fine for months before, through multiple previous updates. And I too discovered that running the CombineAllChunks then then jobx_MuxFiles scripts manually will complete the process. Please fix.

Make sure you're on core 2019.10.06 and you should be good to go. I suspect you're on an older version.

[LigH]

should I ignore it?:

Possibly yes. Your x264 build does not support hardware acceleration (if there is a speed-up at all), but that doesn't keep it from finishing its job correctly with usual CPU routines only.

There may be x264 builds out there which can use some OpenCL functions of a modern GPU. So you might replace xour x264 executable; but I won't guarantee that they work correctly with RipBot264, they might interfere with other filters using the GPU too, even. Test and get experienced.

[mparade]

Thank you very much.

DTS-HD MA 5.1 audio track was identified as 2.0 by Ripbot.
After reading input stream only stereo profiles were available from the list.

[Dhry]

Make sure you're on core 2019.10.06 and you should be good to go. I suspect you're on an older version.

That was correct. However, this morning it auto-updated, I tried a test 265 encode and it worked fine. Cheers!

[byteshare]

That was correct. However, this morning it auto-updated, I tried a test 265 encode and it worked fine. Cheers!
You should re-create the jobs if it just updated (the ones that haven't been processed).

[Dhry]

You should re-create the jobs if it just updated (the ones that haven't been processed).

No need. For the jobs that died, the folders are still there and all I need to do is to manually run the two existing scripts to combine and remux and all's well.

[Dhry]

Feature request: I have a 4-server distributed rendering setup. I notice that when a job is near the end, two of my machines remain idle (having completed their assigned work) while the last two (or one) complete their render pieces. In order to avoid servers being idle for periods of time like this, could we have the next job start while servers in the farm are still completing the last chunks of the previous job, so that no servers are idle at any time?

Cheers
Dhry

[Atak_Snajpera]

Feature request: I have a 4-server distributed rendering setup. I notice that when a job is near the end, two of my machines remain idle (having completed their assigned work) while the last two (or one) complete their render pieces. In order to avoid servers being idle for periods of time like this, could we have the next job start while servers in the farm are still completing the last chunks of the previous job, so that no servers are idle at any time?

Cheers
Dhry

No... Too much hassle ,too little benefit...

[byteshare]

You should re-create the jobs if it just updated (the ones that haven't been processed).

No need. For the jobs that died, the folders are still there and all I need to do is to manually run the two existing scripts to combine and remux and all's well.

The jobs that haven't run should be re-created but you can roll the dice ;)

[byteshare]

Feature request: I have a 4-server distributed rendering setup. I notice that when a job is near the end, two of my machines remain idle (having completed their assigned work) while the last two (or one) complete their render pieces. In order to avoid servers being idle for periods of time like this, could we have the next job start while servers in the farm are still completing the last chunks of the previous job, so that no servers are idle at any time?
If the computers are similar in performance you should try having the jobs split into 1 part per server/encoding server and set the priority to variable.
Or, have a 2nd application on each computer encoding a low priority video.
The feature was something I thought I wanted a while ago but with faster computers now I'm only seeing one to a few min. of lost time which only would add up if I was doing this for years and profit.

[Atak_Snajpera]

Variable priority makes only sense If you run multiple encoding servers on single pc. (For example to get 100% CPU usage)

[byteshare]

Variable priority makes only sense If you run multiple encoding servers on single pc. (For example to get 100% CPU usage)
Oh, okay. I remembered it wrong. I haven't used it in a while. So if you have more than 1 machine, not using only 2 chunks, and/or aren't using more than 1 encoding server it won't make sense, got it.
I stopped using it because using only 2 chunks didn't give me a good way to "pause" encodes. I usually stop 1 or more servers to either lower usage or "pause" an encode.

[guest]

Feature request: I have a 4-server distributed rendering setup. I notice that when a job is near the end, two of my machines remain idle (having completed their assigned work) while the last two (or one) complete their render pieces. In order to avoid servers being idle for periods of time like this, could we have the next job start while servers in the farm are still completing the last chunks of the previous job, so that no servers are idle at any time?

Cheers
Dhry

No... Too much hassle ,too little benefit...

I have to agree with this request....

I to have a multi pc server farm, and I get very frustrated at the wasted resources between encodes...if you are encoding a 4K movie, when it gets to the muxing stage, ONLY 1 pc is working on that, ALL the other's a twiddling their thumbs, for an extended period, could be 20 - 30 minutes !!! So over several movies in a queue, that's hour's wasted !!!

Just because you don't have a set up like this, you think it's a waste of time.

[guest]

How do we use the "Run Script after finished job"?

What do you mean?

Just check Job finished successfully.cmd and Job finished with error.cmd

I don't see an option/where to define what script it will run, unless it is an internal script?
I can check it but I don't see what it does, so I don't know how to use it.
--Oh, thank you for adding GPU as a option in the settings for decoders and not just a right click option. Other than needing Intel or NVidia to use, any downsides to it over FFMS2 or LSMASH?

Also, you add these new features, and don't give any explanation on what they are for, or how to enable/use them....

And what about us users that have AMD GPU's ???

And here's another feature request..

How about displaying the Core version on the main gui page, so it's there for everyone to see, instead of looking for it.

And another suggestion (that has been asked for before), wouldn't it be beneficial to change the build number by a .1 for EVERY auto update, that too would help keep track of build's/version's.

And while I'm at it, a changelog would be good, too.

[Atak_Snajpera]

I have to agree with this request....

I to have a multi pc server farm, and I get very frustrated at the wasted resources between encodes...if you are encoding a 4K movie, when it gets to the muxing stage, ONLY 1 pc is working on that, ALL the other's a twiddling their thumbs, for an extended period, could be 20 - 30 minutes !!! So over several movies in a queue, that's hour's wasted !!!

Just because you don't have a set up like this, you think it's a waste of time.

I'm honestly surprised that muxing in your case takes 30 min. On cheap sata3 SSD muxing 10 GiB files should take 1-2 minutes.

.And what about us users that have AMD GPU's ???
AMD despite my sympathy is still a dick and does not allow 3rd party developers access to their video engine. For example NVIDIA Has cuvid API and Intel Has quicksync.

[guest]

I'm honestly surprised that muxing in your case takes 30 min. On cheap sata3 SSD muxing 10 GiB files should take 1-2 minutes.

OMG, well clearly you don't have much experience with high bitrate 4K muxing !!

10Gb for a 4K movie !!, more like 60 - 80Gb !!

Admittedly I'm not using SSD's, only 7200rpm spinners.

[Atak_Snajpera]

OMG, well clearly you don't have much experience with high bitrate 4K muxing !!

10Gb for a 4K movie !!, more like 60 - 80Gb !!

Admittedly I'm not using SSD's, only 7200rpm spinners.

OMG, I'm sorry for not reading your mind correctly! Unfortunately my crystal ball is currently under inspection. Maybe it is time to buy some crappy sata3 ssd just for muxing? Assuming that copying speed on sata3 ssd is around 0.25GiB/s ,your big ass 80 GiB file would be muxed in ~5 minutes. Furthermore on decent NVMe SSD this would be reduced to ~1 minute.

[guest]

OMG, I'm sorry for not reading your mind correctly! Unfortunately my crystal ball is currently under inspection. Maybe it is time to buy some crappy sata3 ssd just for muxing? Assuming that copying speed on sata3 ssd is around 0.25GiB/s ,your big ass 80 GiB file would be muxed in ~5 minutes. Furthermore on decent NVMe SSD this would be reduced to ~1 minute.

Well, my crystal ball doesn't seem to work, and my mind reading isn't up to scratch, either.

I will dig out a "crappy ssd", and try a "big ass 80Gb" encode, with the latest build of RB, and see just how long it does take....

[Atak_Snajpera]

I will dig out a "crappy ssd", and try a "big ass 80Gb" encode, with the latest build of RB, and see just how long it does take....
If it still takes long time then I will rethink your request. Just give me some hard numbers.

[SKPN]

I'm having two issues with my latest set of encodes. The source, according to MediaInfo, has a 1s audio delay relative to the video. This delay is not being copied over, causing the audio to be slightly ahead of the video track. Is there a way for the timing to be picked up automatically by RipBot?

I've tried using the DelayAudio option in the AVS script, but that didn't seem to work, and I would ideally like to not have to do anything custom because I have over a hundred videos to do with the same issue.

The second issue is that the aspect ratio on the encoded video is incorrect. The source videos are 1440x1080 with a 4:3 DAR. For some reason, the encoded videos are being given a DAR of 16:9. I'm not using any custom scripts or anything that would change this, and I can't figure out why it's not just using the source DAR.

For this, I've tried using --sar 4:3 in the encoding settings, but the DAR is still 16:9 in the resulting video. I've also tried using a custom resize option, setting it to 1440 width with a 1.33:1 ratio.

Any help would be greatly appreciated.

[sneaker_ger]

1440x1080 with 4:3 DAR means --sar 1:1

(sar is sample aspect ratio)

[Atak_Snajpera]

For this, I've tried using --sar 4:3 in the encoding settings, but the DAR is still 16:9 in the resulting video. I've also tried using a custom resize option, setting it to 1440 width with a 1.33:1 ratio.
You should use --sar 1:1

[SKPN]

1440x1080 with 4:3 DAR means --sar 1:1

(sar is sample aspect ratio)
You should use --sar 1:1

Thank you! I don't know why I couldn't figure that out. Is there any reason it wouldn't do that automatically though? I'm not sure why it was setting it to 16:9 by default when the source video is 4:3.

In any case, that's one issue solved. Any suggestions for the audio sync issue by chance?

[Atak_Snajpera]

Do you re-encode audio or just remux? How many audio sources does your file have?

[SKPN]

Do you re-encode audio or just remux? How many audio sources does your file have?

Just remuxed, and just one audio source. I did figure out how to fix it for my one test video, but it involved a manual edit for the audio delay. That would normally be a good enough solution, but I have more than a hundred videos, and if the delay isn't the same on all of them, I'll have to go through each one individually to fix them unless I can figure out a way to have it read that delay by default.

[Atak_Snajpera]

Is delay always 1000ms?

[SKPN]

Is delay always 1000ms?

That I'm not sure of. It is on the few I've spot-checked, so I may be able to just run it with that delay, and fix the ones I come across that are wrong, but ideally I would be able to just have it use the same settings that are in the source file.

One other question: When doing a batch, is there a way to select which subtitles to render? When doing individual encodes, I can obviously select which subtitle track I want to burn in, but when I tested with the batch option on a file that had two subtitle tracks, it didn't burn either into the video; it just dropped them both from the resulting file altogether.

[Atak_Snajpera]

One other question: When doing a batch, is there a way to select which subtitles to render? When doing individual encodes, I can obviously select which subtitle track I want to burn in, but when I tested with the batch option on a file that had two subtitle tracks, it didn't burn either into the video; it just dropped them both from the resulting file altogether.
subtitle must be in the same folder as video file and have the same name to be automatically burned in. For example
C:\Folder\video.mkv
C:\Folder\video.srt

[byteshare]

I to have a multi pc server farm, and I get very frustrated at the wasted resources between encodes...if you are encoding a 4K movie, when it gets to the muxing stage, ONLY 1 pc is working on that, ALL the other's a twiddling their thumbs, for an extended period, could be 20 - 30 minutes !!! So over several movies in a queue, that's hour's wasted !!!
Just because you don't have a set up like this, you think it's a waste of time.
As already pointed out using an SSD as your mux drive (temp drive and end point for encodes) would speed things up, but you could alternatively have a lower priority encode with HandBrake or something on each to fill that time. (more to manage I know)
Also, you add these new features, and don't give any explanation on what they are for, or how to enable/use them....
And what about us users that have AMD GPU's ???
And here's another feature request..
How about displaying the Core version on the main gui page, so it's there for everyone to see, instead of looking for it.
And another suggestion (that has been asked for before), wouldn't it be beneficial to change the build number by a .1 for EVERY auto update, that too would help keep track of build's/version's.
And while I'm at it, a changelog would be good, too.
-While you can't use AMD GPUs for decode you can use them for some filters that use OpenCL GPU, such as KNLMeansCL.
-I would like a better way of knowing which version I'm actually on and what was changed but that is up to the dev so I understand if they don't want to spend more time/energy changing the version every release or changing the way the release is shown (with the core version) and giving detailed notes of the changes. I've become accustomed to looking for changes or asking and I almost always get a response.
OMG, well clearly you don't have much experience with high bitrate 4K muxing !!
Maybe it is just me but this comes off a little hostile. This is a free program and it is both the encoding app I use mostly and my preferred--I would say it isn't perfect but the best for experienced users that want AVISynth.

[byteshare]

That I'm not sure of. It is on the few I've spot-checked, so I may be able to just run it with that delay, and fix the ones I come across that are wrong, but ideally I would be able to just have it use the same settings that are in the source file.
If you mux in the audio from the source (manually) to the encoded file does it keep the timing? I can't remember if it does.
If it does, you could skip audio in RB and just use a bactch file to copy the audio at the end to all of your encoded files from the source files.

[SKPN]

subtitle must be in the same folder as video file and have the same name to be automatically burned in. For example
C:\Folder\video.mkv
C:\Folder\video.srt

The subtitles are embedded in the source file. Do I need to extract and rename them for it to work properly?

If you mux in the audio from the source (manually) to the encoded file does it keep the timing? I can't remember if it does.
If it does, you could skip audio in RB and just use a bactch file to copy the audio at the end to all of your encoded files from the source files.

I will give that a try.

Edit: It does keep the original timing if I mux the source audio in manually. I guess I will need to figure out how to write a batch file to extract the audio from the source and mux it into the encoded file.

[Atak_Snajpera]

The subtitles are embedded in the source file. Do I need to extract and rename them for it to work properly?
1) Run batch mode but select Prepare all jobs and wait for user
2) Demuxed subtitles should be in Job folder. Now you will have to copy selected subtitles to folder like I mentioned in previous post
3) Run again batch mode.

I guess I will need to figure out how to write a batch file to extract the audio from the source and mux it into the encoded file.
If you are familiar with batch scripting then you could remux encoded file with correct audio delay using Job finished successfully.cmd

[SKPN]

If you are familiar with batch scripting then you could remux encoded file with correct audio delay using Job finished successfully.cmd

How do I make use of the Job Finished Successfully cmd? Where is that located? I was able to create a batch file to fix the audio delay, but I'm not familiar with post-processing with RipBot.

I also solved my subtitle problem by just extracting them with MKVCleaver and doing a bulk rename to match the originals, so thank you for your help there!

[Atak_Snajpera]

If you have latest version then it should be located in the same folder as RipBot264.exe

[byteshare]

Edit: It does keep the original timing if I mux the source audio in manually. I guess I will need to figure out how to write a batch file to extract the audio from the source and mux it into the encoded file.
I haven't figured out "Job finished successfully.cmd" completely but also I have issues with names matching up since RipBot takes out the "." (I know I could manually remove it on my sources first, but haven't yet)
That said, but this in a "File.bat" and adjust the directories as needed (Assuming your MKVMerge is where I put it as well):

@echo off
FOR %%A IN (*.mkv) DO "C:\Program Files\MKVToolNix\mkvmerge.exe" -o ".\Muxed\%%A" "%%A" -D -S --no-global-tags "C:\Source\%%A"
pause

You'd run the batch file where the output (encoded file) is.
If you use the "Job finished successfully.cmd" you could have this be "automated", downside would be it would remux all previous encoded files unless you had those automatically deleted/moved.

[Atak_Snajpera]

I haven't figured out "Job finished successfully.cmd" completely
It is very simple. Let's say you want to delete original file after successful conversion.
This is how to do it.

@REM *********** SUPPORTED EXTRA VARIABLES ***********
@REM * *
@REM * %RIPBOT264TEMP% = e.g. c:\Temp\Ripbot264temp *
@REM * %JOB% = e.g. Job1 *
@REM * %INPUTFILEDIR% = e.g. C:\SourceVideos *
@REM * %OUTPUTFILEDIR% = e.g. C:\EncodedVideos *
@REM * %INPUTFILENAME% = e.g. MySourceVideo *
@REM * %OUTPUTFILENAME% = e.g. MyEncodedVideo *
@REM * %INPUTFILEEXT% = e.g. avi *
@REM * %OUTPUTFILEEXT% = e.g. mkv *
@REM * %YEAR% = e.g. 2019 *
@REM * %MONTH% = e.g. 01 *
@REM * %DAY% = e.g. 31 *
@REM * %HOUR% = e.g. 23 *
@REM * %MINUTE% = e.g. 59 *
@REM * %SECOND% = e.g. 00 *
@REM * %MILLISECOND% = e.g. 999 *
@REM * *
@REM *************************************************
del %INPUTFILEDIR%\%INPUTFILENAME%.%INPUTFILEEXT%

[SKPN]

If you have latest version then it should be located in the same folder as RipBot264.exe

That would explain why I can't find it. I'm still on v1.24.1. I remember trying out v1.25.0 and it being considerably slower than 1.24, so I went back.

[byteshare]

It is very simple. Let's say you want to delete original file after successful conversion.
This is how to do it.

@REM *********** SUPPORTED EXTRA VARIABLES ***********
@REM * *
@REM * %RIPBOT264TEMP% = e.g. c:\Temp\Ripbot264temp *
@REM * %JOB% = e.g. Job1 *
@REM * %INPUTFILEDIR% = e.g. C:\SourceVideos *
@REM * %OUTPUTFILEDIR% = e.g. C:\EncodedVideos *
@REM * %INPUTFILENAME% = e.g. MySourceVideo *
@REM * %OUTPUTFILENAME% = e.g. MyEncodedVideo *
@REM * %INPUTFILEEXT% = e.g. avi *
@REM * %OUTPUTFILEEXT% = e.g. mkv *
@REM * %YEAR% = e.g. 2019 *
@REM * %MONTH% = e.g. 01 *
@REM * %DAY% = e.g. 31 *
@REM * %HOUR% = e.g. 23 *
@REM * %MINUTE% = e.g. 59 *
@REM * %SECOND% = e.g. 00 *
@REM * %MILLISECOND% = e.g. 999 *
@REM * *
@REM *************************************************
del %INPUTFILEDIR%\%INPUTFILENAME%.%INPUTFILEEXT%


Ah, okay. I thought maybe I needed to define the extra variables. I thought they were all examples of what they could equal. Didn't realize they were pulling information such as the %INPUTFILEDIR% from the job itself.

[byteshare]

That would explain why I can't find it. I'm still on v1.24.1. I remember trying out v1.25.0 and it being considerably slower than 1.24, so I went back.
Two things have changed since then that might have been why it was slow for you:
1. LSmash doesn't have to be default
2. there was a fix for using some of the filters with multi-threading

[byteshare]

Would something like this work for the "Job finished successfully.bat":
"C:\Program Files\MKVToolNix\mkvmerge.exe" -o "%OUTPUTFILEDIR%\Muxed\%INPUTFILENAME%.mkv" "%OUTPUTFILEDIR%\%OUTPUTFILENAME%.%OUTPUTFILEEXT%" -D -S --no-chapters --no-global-tags "%INPUTFILEDIR%\%INPUTFILENAME%.%INPUTFILEEXT%"
timeout 5
MOVE %OUTPUTFILEDIR%\%OUTPUTFILENAME%.%OUTPUTFILEEXT% "%OUTPUTFILEDIR%\Hold"
Did the last MOVE rather than del encase something went wrong or maybe someone would still want it until they verified the output (Muxed) folder.

Will RipBot be waiting to do the next job while the "Job finished successfully.bat" is done running?

[Atak_Snajpera]

.Will RipBot be waiting to do the next job while the "Job finished successfully.bat" is done running?

No...

[guest]

As already pointed out using an SSD as your mux drive (temp drive and end point for encodes) would speed things up,

I will definitely be trying that...

-I would like a better way of knowing which version I'm actually on and what was changed but that is up to the dev so I understand if they don't want to spend more time/energy changing the version every release or changing the way the release is shown (with the core version) and giving detailed notes of the changes. I've become accustomed to looking for changes or asking and I almost always get a response.

I'm sure it would only take a couple of minutes..

Maybe it is just me but this comes off a little hostile. This is a free program and it is both the encoding app I use mostly and my preferred--I would say it isn't perfect but the best for experienced users that want AVISynth.

I know that my reply might seem a bit like that, but it was in response to the previous post, which was rather irrelevant to high bitrate 4K content. I do recall that the dev mentioned (albeit a while back) that he had no, to limited access to 4K material.

Thank you for your input :)

[byteshare]

No...
Awesome. I was worried that it would slow down jobs.
I'll run some tests on it now. I have the "keep jobs after successful conversion" not checked. Need to see if the job folder is removed before or after the script because I was realizing that I could copy most of what i need from the video.mkv directly since it still have audio, subs, and things and is on my SSD scratch drive already rather than the physical HD I use for storing sources.

[Atak_Snajpera]

Awesome. I was worried that it would slow down jobs.
I'll run some tests on it now. I have the "keep jobs after successful conversion" not checked. Need to see if the job folder is removed before or after the script because I was realizing that I could copy most of what i need from the video.mkv directly since it still have audio, subs, and things and is on my SSD scratch drive already rather than the physical HD I use for storing sources.

Like I said code in ripbot does not wait for cmd script so job folder will be deleted anyway. I predict that your script will block files and you will end up with some files still present in job folder.

[duffbeer]

I'm still seeing very slow performance when setting up an encode since the update to 1.25 and none of the small updates since have solved the problems for me.
It takes around 20 mins to demux and index my source and then another 10 mins to use the auto crop. After the auto crop has finished it takes another 10 mins to return to the main menu where I can start the job queue.

40+ mins for each job is really frustrating and I can't see any obvious reason that this has now changed. If I go back to 1.24.1, everything works within a few seconds.

Does anyone have any idea what may be happening here?

[forkicks]

Hello.

I am running rb updated (just did a recheck for updates after running rb for 3 days).

I got this on a few jobs (with different files):
----------[2019-10-09 20:54:26]----------

File: Y:\batch1\les.sauvages.s01e01.french.1080p.hdtv.x264-hybris.mkv
FFAudioSource: Audio format change detected. This is currently unsupported. Channels: 6 -> 2; Sample rate: 48000 -> 48000; Sample format: fltp -> fltp
(C:\Temp\RipBot264temp\job16\getinfo.avs, line 4)


Most of the files encoded properly, but these few failed with this error. The thing is, in my job definition, i have Rb set to copy the audio track, not encode it at all, so i don't understand why it's trying to check this.

Thanks,
fK

[Atak_Snajpera]

Your file has not been correctly encoded. Most likely there is still audio related with advertisements. You need to manually cut out those left overs.

[Atak_Snajpera]

I'm still seeing very slow performance when setting up an encode since the update to 1.25 and none of the small updates since have solved the problems for me.
It takes around 20 mins to demux and index my source and then another 10 mins to use the auto crop. After the auto crop has finished it takes another 10 mins to return to the main menu where I can start the job queue.

40+ mins for each job is really frustrating and I can't see any obvious reason that this has now changed. If I go back to 1.24.1, everything works within a few seconds.

Does anyone have any idea what may be happening here?

Try again in safe mode without any antiviruses active and other apps.

[byteshare]

I'm still seeing very slow performance when setting up an encode since the update to 1.25 and none of the small updates since have solved the problems for me.
It takes around 20 mins to demux and index my source and then another 10 mins to use the auto crop. After the auto crop has finished it takes another 10 mins to return to the main menu where I can start the job queue.

40+ mins for each job is really frustrating and I can't see any obvious reason that this has now changed. If I go back to 1.24.1, everything works within a few seconds.

Does anyone have any idea what may be happening here?
Did you change the decoder from LSmash to FFMS2?

[byteshare]

Like I said code in ripbot does not wait for cmd script so job folder will be deleted anyway. I predict that your script will block files and you will end up with some files still present in job folder.
So, the problem I'm seeing now (changed things up) is once the job is removed RB loses the ability to use the variables such as %INPUTFILENAME%
example:
MOVE "%INPUTFILEDIR%\%INPUTFILENAME%.mkv" "C:\done\%INPUTFILENAME%.mkv"
When the script runs it doesn't have an input name. The work around is to not automatically delete jobs when successful.
Would it be possible for the next job to start but not delete the job folder until after the script runs?
Or is there a good way to automatically remove a job via the bat file, and by not just deleting the %Job% folder?

[Atak_Snajpera]

Would it be possible for the next job to start but not delete the job folder until after the script runs?
I will think about it... Shouldn't be too difficult to implement.

[slalom]

After yesterdays Windows Update for 7, I get an error

List index out of bounds (130)

I press OK, and the program opens and some jobs are missing, but they are in my Temp folder

It's not the app, I unzipped and updated the files in the first post, same thing

[duffbeer]

Try again in safe mode without any antiviruses active and other apps.

I'll give it a go but I only have standard Windows 10 Anti Virus and no other apps running. This is a dedicated encoding PC.

[duffbeer]

Did you change the decoder from LSmash to FFMS2?

Yes I had to. LSMASH makes it even slower.

[Atak_Snajpera]

After yesterdays Windows Update for 7, I get an error

List index out of bounds (130)

I press OK, and the program opens and some jobs are missing, but they are in my Temp folder

It's not the app, I unzipped and updated the files in the first post, same thing

Delete job.list file

[byteshare]

I will think about it... Shouldn't be too difficult to implement.
Thank you, and if you think of a better solution I'm all for it. Just the things I thought about in the moment.

Yes I had to. LSMASH makes it even slower.
Is your Temp drive for RipBot an SSD? I use a physical drive for storing my sources and an SSD for the Temp drive (jobs) and adding new jobs isn't that slow for me on the latest version.

[guest]

Is your Temp drive for RipBot an SSD? I use a physical drive for storing my sources and an SSD for the Temp drive (jobs) and adding new jobs isn't that slow for me on the latest version.

Speaking of taking time...it took 27 minutes to "prepare" a 4K movie, copying from a spinner (main movie) to an SSD temp drive, (just like you, byteshare) with Lsmash.

It was John Wick 3, @ approx 65 Gb.

And yes, I am yet to complete my "muxing" test, that will be in a few days, will be interesting if SSD's make much difference to large files.

[Atak_Snajpera]

I do not get IT. Are you encoding 60gib movie to 60gib file? In your earlier post you mentioned that muxing takes long time because your final encoded file was 60-80gib.

[guest]

I do not get IT. Are you encoding 60gib movie to 60gib file? In your earlier post you mentioned that muxing takes long time because your final encoded file was 60-80gib.

I didn't think you would, but this particular movie is approx 65Gb, has a 63.5Mb/s bitrate.

I simply want to strip stuff out, redo audio, as most 4K movies are TrueHD, and my tv doesn't support that. So I have to convert it either with RB, or eac3to.

Also some filtering, subtitling, etc, if required.

So I am aiming to do the majority of the 4K movies I have @ a 60Mb/s bitrate, that will increase some, and decrease others.

So I do things differently to most...so what !!!!

[LigH]

So you want to do more than just remultiplexing.

I simply want to strip stuff out, redo audio, as most 4K movies are TrueHD, and my tv doesn't support that. So I have to convert it either with RB, or eac3to.

Up to this point, the video may be left unchanged, all operations can be fast.

Also some filtering, subtitling, etc, if required.

This possibly requires recoding the video (especially if "subtitling" means permanently encoding = "burning" subtitles into the video stream). That will take a lot of time in 4K resolution.

[guest]

So you want to do more than just remultiplexing.

Up to this point, the video may be left unchanged, all operations can be fast.

This possibly requires recoding the video (especially if "subtitling" means permanently encoding = "burning" subtitles into the video stream). That will take a lot of time in 4K resolution.

I think I will use RB to change the bitrate, but use other software to convert the TrueHD audio, I also have to convert subtitles to .srt (because that's what my tv requires), so use Subtitle Edit to do that.

Unfortunately, RB can't convert TrueHD to DTS, or subtitles to .srt :(

Once the audio & sub's have been converted, and RB has done it's thing on the video (only), will reconstruct it all back together with MKVToolnix.

Admittedly, if I left the original bitrate, then the process would be a little different..

A LOT of work, but that's what I want.

[slalom]

Why don't you buy an android TV box and play the file with Kodi?

Delete job.list file
Nothing happened, same error

[guest]

Why don't you buy an android TV box and play the file with Kodi?

Been there, done that....not impressed !!!!

[SKPN]

I didn't think you would, but this particular movie is approx 65Gb, has a 63.5Mb/s bitrate.

I simply want to strip stuff out, redo audio, as most 4K movies are TrueHD, and my tv doesn't support that. So I have to convert it either with RB, or eac3to.

Also some filtering, subtitling, etc, if required.

So I am aiming to do the majority of the 4K movies I have @ a 60Mb/s bitrate, that will increase some, and decrease others.

So I do things differently to most...so what !!!!

I'm confused at why you are re-encoding the video at all based on what you're describing. Why not just extract the audio and subtitles, convert them using the other applications you mentioned, then mux them back in with the original video? Even if you want to cut out some fluff like credits, you can do that without re-encoding the video. If you're not rendering the subtitles and you're not trying to compress the video, what is the purpose of re-encoding it?

[slalom]

Been there, done that....not impressed !!!!
Did you try an HTPC too?

[guest]

I'm confused at why you are re-encoding the video at all based on what you're describing. Why not just extract the audio and subtitles, convert them using the other applications you mentioned, then mux them back in with the original video? Even if you want to cut out some fluff like credits, you can do that without re-encoding the video. If you're not rendering the subtitles and you're not trying to compress the video, what is the purpose of re-encoding it?

Admittedly, some 4K movies could be done an easier way, but on some where I want to increased (or decrease) the bitrate, I have to re encode it.

Not trying to compress !!, mind you, I really haven't looked into that, too much.

[guest]

Did you try an HTPC too?

No I haven't !!

HTPC and/or streaming doesn't suit me, I don't like the idea of having a pc running, just to watch movies.

[SKPN]

Admittedly, some 4K movies could be done an easier way, but on some where I want to increased (or decrease) the bitrate, I have to re encode it.

Not trying to compress !!, mind you, I really haven't looked into that, too much.

Why would you want to increase the bitrate?

[Atak_Snajpera]

Why would you want to increase the bitrate?

Because increasing bitrate also automagically increases quality ;) doh

[byteshare]

Speaking of taking time...it took 27 minutes to "prepare" a 4K movie, copying from a spinner (main movie) to an SSD temp drive, (just like you, byteshare) with Lsmash.

It was John Wick 3, @ approx 65 Gb.

And yes, I am yet to complete my "muxing" test, that will be in a few days, will be interesting if SSD's make much difference to large files.
I almost never use LSmash, so I don't see those delays with large files.
Sorry. Try FFMS2 or GPU decode (if applicable). The LSmash slowness issues isn't RB's fault.

I didn't think you would, but this particular movie is approx 65Gb, has a 63.5Mb/s bitrate.

I simply want to strip stuff out, redo audio, as most 4K movies are TrueHD, and my tv doesn't support that. So I have to convert it either with RB, or eac3to.

Also some filtering, subtitling, etc, if required.

So I am aiming to do the majority of the 4K movies I have @ a 60Mb/s bitrate, that will increase some, and decrease others.

So I do things differently to most...so what !!!!
Try Foobar2000 if you just want to process audio.
Are you encoding to HEVC/x265? If so your files should be smaller and still look the same even if you're using CRF18.
As pointed out, it sounds like you're doing a lot of work for just getting your TV to play a file when you wouldn't have to do all of that if you got a stand alone set-top box like an Nvida Shield TV (don't need a PC to use Plex) or Mi Box (~$50 USD).

Because increasing bitrate also automagically increases quality ;) doh
LOL

[mparade]

According to Windows scheduler it seems that Ripbot is not able to saturate my 1950X even in distributed encoding mode.
Processor utilization fluctuates between 78-85% with any number of encodes started.Ripbot sees only Numa node 0 with 32 threads instead of 0 and 1 with 16 each. Is this the problem?

Interestingly, my 2990WX is fully saturated with it's 32 cores while all 4 nodes being correctly identified by Ripbot.

Any help would be appreciated.

[Atak_Snajpera]

1950x by default works in Uma mode (can be changed in bios) while 2990wx only supports NUMA mode.

[mparade]

1950x by default works in Uma mode (can be changed in bios) while 2990wx only supports NUMA mode.

Thanks.

In UMA mode my 1950x seems to be like "turtle" compared to 2990wx. Is it possible that 1950x is not correctly recognized by Ripbot? My 2990wx is recognized as "AMD-2990wx" in distributed encoding window while my 1950x is not as something like "AMD-1950x". If I change my 1950x to NUMA mode Ripbot tells it still has only one node. I am going to play with my 1950x tomorrow to find out why it is slow and cannot saturate the cores using Ripbot.

[Atak_Snajpera]

Does task manager also see single node?

[mparade]

Sorry for the question but where is this presented in task manager? :)

Anyway, it is like magic "transcoding" movies with "placebo" settings using Ripbot! :)

[Atak_Snajpera]

Right Click on CPU graph and select NUMA nodes.

[mparade]

Now both Windows task manager and Ripbot shows 2 nodes...sorry...I must have been drunk.
Now I am on 100% with my 1950x as well...just need to enter NUMA mode. Thank you.

[Atak_Snajpera]

Now both Windows task manager and Ripbot shows 2 nodes...sorry...I must have been drunk.
Now I am on 100% with my 1950x as well...just need to enter NUMA mode. Thank you.

I suppose that intensive (latency/bandwith)communication between dies prevents you from 100% CPU usage. In NUMA mode both dies work on separated data.

[guest]

Because increasing bitrate also automagically increases quality ;) doh

Ha, so sarcasm aside...prove me wrong !!!

Someone provide me with some proven 4K settings to equal original video quality, at a much lower bitrate & file size !!

[byteshare]

Ha, so sarcasm aside...prove me wrong !!!

Someone provide me with some proven 4K settings to equal original video quality, at a much lower bitrate & file size !!
Well, more bitrate than the original doesn't mean better quality, since there are plenty of ways to increase a bitrate but still get garbage quality. When dealing with x264/x265 there are a lot of complicated settings you can tweak, but usually either you're going for 1:1 quality with the original or some balance of quality with size. I prefer the later. If I want 1:1 I'll just copy the video and not process it.
I've encoded enough video to know there is no perfect settings for all videos of all types. You can get good general settings but for better results you'll need to tweak settings per video.

[guest]

Well, more bitrate than the original doesn't mean better quality, since there are plenty of ways to increase a bitrate but still get garbage quality. When dealing with x264/x265 there are a lot of complicated settings you can tweak, but usually either you're going for 1:1 quality with the original or some balance of quality with size. I prefer the later. If I want 1:1 I'll just copy the video and not process it.
I've encoded enough video to know there is no perfect settings for all videos of all types. You can get good general settings but for better results you'll need to tweak settings per video.

So I have been doing some more "research", and it seems that a bitrate of around 55 - 60Mb/s is the high end of 4K HEVC encodes.

Some, are higher, and some are lower.

With some of my recent encodes I have used a 1 pass with a bitrate set @ 60,000kbp/s (in the RB setting's).

Some might use a CRF (CQ) number of say 18 or 16 to get a good result.

With maybe a Slow preset...I'm not using any preset's at this stage.

So not knowing what your experience is with 4K movies, can you suggest a starting point ??

[byteshare]

So I have been doing some more "research", and it seems that a bitrate of around 55 - 60Mb/s is the high end of 4K HEVC encodes.

Some, are higher, and some are lower.

With some of my recent encodes I have used a 1 pass with a bitrate set @ 60,000kbp/s (in the RB setting's).

Some might use a CRF (CQ) number of say 18 or 16 to get a good result.

With maybe a Slow preset...I'm not using any preset's at this stage.

So not knowing what your experience is with 4K movies, can you suggest a starting point ??
As for general settings, presets aren't a bad place start depending on your needs. I don't use straight presets myself because I want a balanced encode and no preset can keep reasonable quality and size and speed for my tastes.
It isn't clear what you're trying to do to me. If I were to guess it just seems like you want to keep the quality the same as the source, no?
If you want that and aren't doing filters than I'd just use Ultra-Fast downscale a lot and then at the end copy back in the source video.
If you want to keep quality close to the source but still using filters, which filters are you using? Once you're using filters it really changes things. I tend to not like grain so I reduce that with some careful settings.

[guest]

As for general settings, presets aren't a bad place start depending on your needs. I don't use straight presets myself because I want a balanced encode and no preset can keep reasonable quality and size and speed for my tastes.
It isn't clear what you're trying to do to me. If I were to guess it just seems like you want to keep the quality the same as the source, no?
If you want that and aren't doing filters than I'd just use Ultra-Fast downscale a lot and then at the end copy back in the source video.
If you want to keep quality close to the source but still using filters, which filters are you using? Once you're using filters it really changes things. I tend to not like grain so I reduce that with some careful settings.

Well, as I said, I'd like to have my 4K's at a "standard" of 60Mps, and again, as I've said, some have lower than that, some are higher, so for that I need to run them thru RB, even if it's video only.

For what I want & need to do to audio, can be dealt with other app's, as too, the subtitles.

I have to agree, I don't like grain either, but fortunately most 4K's aren't very "noisy", so not a lot of filtering required.

I know I need to try a few different options, but it's the time factor that's my main concern...imagine doing a high bitrate 4K HEVC, on a Slow preset, using MDGrain2....day's :( even with my "farm".

But the other time waster is between when the encode is completed, and the start of the next job in queue...meaning the muxing, etc...it leaves a lot of resources doing nothing. :(, but having said that, I am yet to test this again, after putting some SSD's in the mix.

One thing I would like to get around to doing is upscaling some 1080p to 4K, but do any filtering, etc, still @ 1080p, then once satisfied, do a 4K HEVC encode, and see how it turns out.

But just imagine the time & resources involved when 8K is a lot more common place.

[skylinekiller]

I am trying to edit the font when I hardcode subtitle into my encode, but it doesn't seem to be working. I edited the notepad file in Tools\AviSynth plugins\VSFilter\Default.style with the parameters below.

Style: Default,Univers CE 45 Light,41,16777215,0,16777215,0,-1,0,2,1,2,20,20,20,0,0

what it looks like with the above settings in RipBot https://ibb.co/S7xFvpn


This is what it should look like.. The resolution is not as good but trust me, the font is not accepting the setttings
https://ibb.co/92VB0Xt


This is the setting from srt2ssa

Synch Point: Side 1 0m00s
Collisions: Normal
Timer: 100.0000
Style: Default,Univers CE 45 Light,41,16777215,0,16777215,0,-1,0,2,1,2,20,20,20,0,0


Why am I getting a shadow

[mparade]

I suppose that intensive (latency/bandwith)communication between dies prevents you from 100% CPU usage. In NUMA mode both dies work on separated data.

Under heavy load my Windows on 2990WX used to reset itself randomly without any error message in ripbot however my temperatures are ok in amd ryzen master. 1950x doesn't do it at all even under heavy load...it happens both with encoding 4K and HD.
I am using 4x8GB memory in quad channel layout in both. Power supply units are also powerful enough in both and both are using same type of liquid cooling.
Do you have any suggestion? I would really appreciate that.

[Atak_Snajpera]

Normally when PC shutdowns or resets I blame power supply. Especially when shutdowns/resets happen during heavy load (high power consumption). For testing purposes I would swap power supplies 1950x<->2990wx.

[mparade]

thx!!

[Atak_Snajpera]

Maybe you have activated PBO on 2990wx and hence too high power consumption for your power supply. 0.5kW is just insane.
https://cdn.mos.cms.futurecdn.net/Hx27r3x3yh4r5jJkbiCetS-650-80.png

[mparade]

If I set 8 servers for job 1 in the queue, will the following jobs be using the same servers and threads I assigned to them in job1 automatically for encoding?

[mparade]

Maybe you have activated PBO on 2990wx and hence too high power consumption for your power supply.
https://cdn.mos.cms.futurecdn.net/Hx27r3x3yh4r5jJkbiCetS-650-80.png

Yes, I do (am). :)

[Atak_Snajpera]

All jobs will use the same server configuration.

Yes, I do.
So now you know what to do ;) BTW. Overclocking AMD cpus do not make sense these days.

[mparade]

All jobs will use the same server configuration.

Super-good! Thank you very much!

[mparade]

All jobs will use the same server configuration.


So now you know what to do ;) BTW. Overclocking AMD cpus do not make sense these days.

Does chunk size affect encoding quality more than negligibly? Maybe it is not required at all to set chunk size according to duration of each source.
I would like to saturate my cores using 8 servers almost always that would give very fast encodes.

[Atak_Snajpera]

Does chunk size affect encoding quality more than negligibly?

In CRF mode there is zero quality degradation. In 2-pass degradation is negligible. Also each chunk starts at key-frame.
Use default 1 minute if you are going to use 1950x and 2990wx at the same time.

[mparade]

In CRF mode there is zero quality degradation. In 2-pass degradation is negligible. Also each chunk starts at key-frame.
Use default 1 minute if you are going to use 1950x and 2990wx at the same time.

WOW! I am in CRF mode. Thank you! I am, one from home, the other from workplace…:)

[Atak_Snajpera]

WOW! I am in CRF mode. Thank you! I am, one from home, the other from workplace…:)

With decent internet connection (10Mbps+ upload) and VPN you can use other PC as well.

[mparade]

What is required to use both machines at the same time for the same job? Do you have a tutorial? At the moment I am using them separately with locally set servers.
Please help. Internet connection shouldn't be a problem. Both have Ripbot installed. I can set them up separately now. There is 2 km distance between them. :)

[Atak_Snajpera]

What is required to use both machines at the same time for the same job? Do you have a tutorial? At the moment I am using them separately with locally set servers.
Please help.

You will have to create VPN network on work PC and then connect to that network from home PC. This way shared folder will be visible like on normal LAN. Some users already are using this method to distribute encoding work across internet. It is definitely possible with good internet connection.

https://www.pcworld.com/article/210562/how-set-up-vpn-in-windows.html

[mparade]

Thank you very much!
Source material should be only on home PC, right?
Work PC with VPN network just need to be on with Ripbot installed right? Or should I set somthing more on work pc? Sorry for being noob in this thema.

[Atak_Snajpera]

1) Source files are on your home PC.
2) Work PC just needs EncodingServer.exe running!
3) The idea is that you connect via VPN to work PC and then Work PC sees shared folder (for example \\HOME-PC\RipBot264temp) on Home PC.

[mparade]

I have to accomplish this! I will let you know if get stuck with something (I am sure I will :))

[mparade]

how is the exact number of --seek parameter is specified on x264 command line, may I ask?

thank you for for the help in advance

[Atak_Snajpera]

seek = StartKeyFrameIndex - PreviousKeyFrameIndex

[mparade]

1) Source files are on your home PC.
2) Work PC just needs EncodingServer.exe running!
3) The idea is that you connect via VPN to work PC and then Work PC sees shared folder (for example \\HOME-PC\RipBot264temp) on Home PC.

both work and home pc should be connected via VPN, right?
work pc at the moment cannot because refering to closed port
I have to ask local admin at company

[mparade]

seek = StartKeyFrameIndex - PreviousKeyFrameIndex

thx!!

[Atak_Snajpera]

both work and home pc should be connected via VPN, right?
work pc at the moment cannot because refering to closed port
I have to ask local admin at company

Your admin should know what to do.
https://www.youtube.com/watch?v=Mk_XYuzQ8jA

Ports to open
https://social.technet.microsoft.com/Forums/Azure/en-US/0fff3b48-7615-45ea-817e-5afaad972c43/vpn-ports-to-open?forum=winservergen

[mparade]

All jobs will use the same server configuration.


So now you know what to do ;) BTW. Overclocking AMD cpus do not make sense these days.

There should be something else here...After switching back to auto mode in amd ryzen master, Windows still keeps resetting very frequently right in the middle of encoding...temperature is below 68 degrees so that shouldn't be the problem
maybe memory issue or something else :)

[mparade]

Your admin should know what to do.
https://www.youtube.com/watch?v=Mk_XYuzQ8jA

Ports to open
https://social.technet.microsoft.com/Forums/Azure/en-US/0fff3b48-7615-45ea-817e-5afaad972c43/vpn-ports-to-open?forum=winservergen

thank you very much

[Atak_Snajpera]

thank you very much

This one is even better
https://www.youtube.com/watch?v=eVxHhSLUrxo

[mparade]

All jobs will use the same server configuration.


So now you know what to do ;) BTW. Overclocking AMD cpus do not make sense these days.

Without overclocking it does the same so maybe voltage drop occurs...
so maybe I need to replace the likely faulty 1200W Cooler Master PSU first. :)

[skylinekiller]

so is it no longer possible to change the subtitle fonts.

I am trying to edit the font when I hardcode subtitle into my encode, but it doesn't seem to be working. I edited the notepad file in Tools\AviSynth plugins\VSFilter\Default.style with the parameters below.

Style: Default,Univers CE 45 Light,41,16777215,0,16777215,0,-1,0,2,1,2,20,20,20,0,0

what it looks like with the above settings in RipBot https://ibb.co/S7xFvpn


This is what it should look like. The resolution is not as good but trusts me, the font is not accepting the settings
https://ibb.co/92VB0Xt


This is the setting from srt2ssa

Synch Point: Side 1 0m00s
Collisions: Normal
Timer: 100.0000
Style: Default,Univers CE 45 Light,41,16777215,0,16777215,0,-1,0,2,1,2,20,20,20,0,0


Why am I getting a shadow

[mparade]

Possibly yes. Your x264 build does not support hardware acceleration (if there is a speed-up at all), but that doesn't keep it from finishing its job correctly with usual CPU routines only.

There may be x264 builds out there which can use some OpenCL functions of a modern GPU. So you might replace xour x264 executable; but I won't guarantee that they work correctly with RipBot264, they might interfere with other filters using the GPU too, even. Test and get experienced.

Can anyone recommend a reliable site for x264 builds using OpenCL functions? I have a 2060 RTX so just want to give it a try if possible.

Thank you in advance for the help.

[sneaker_ger]

Try:
https://artifacts.videolan.org/x264/ (no mp4 output - is that a problem for RipBot?)

[mparade]

thx! sorry for the dumb question but if I have Win 64 bit and AMD CPU which version to use AMD64 or Win64 or all the same?

[sneaker_ger]

"release-debian-amd64" is for for the Linux distribution "Debian", not Windows. Try "release-win64" (some software may need "release-win32" for compatibility but win64 is faster and can use more RAM so try that first).

[mparade]

I test is very soon, thank you!

[mparade]

"release-debian-amd64" is for for the Linux distribution "Debian", not Windows. Try "release-win64" (some software may need "release-win32" for compatibility but win64 is faster and can use more RAM so try that first).

Using this version I got the same disabling message from x264: x264 [warning]: OpenCL: not compiled with OpenCL support, disabling

[sneaker_ger]

I tested opencl works with https://artifacts.videolan.org/x264/release-win64/x264-r2984-3759fcb.exe
So you are still using the "wrong" binary somehow.

[mparade]

I have double checked. Ripbot is using the same from it's temp directory. Maybe the commandline made by Ripbot is not ok for the encoder. I don't really know what is happening.

Maybe it has something to do with video decoding? I am using LSMASH.

Thanks for any kind of help.

[LigH]

thx! sorry for the dumb question but if I have Win 64 bit and AMD CPU which version to use AMD64 or Win64 or all the same?

Based on the IA-32-x86 architecture (which was introduced with the intel 80386 CPU), AMD and intel both developed an extension to allow 64 bit addressing. intel first started with a new architecture known as IA-64 (Itanium) which is not compatible with previous software. AMD instead developed a compatible extension x86-64, and intel realized how useful it is to continue using existing 32-bit software while developing 64-bit software with mostly the same instruction set for the future, and developed AMD's x86-64 architecture further with own extensions...

So if you don't have specifically an Itanium Server software for Windows running on an Itanium CPU, which I doubt a lot, then you will have a 64-bit Windows running on a CPU compatible to the AMD64 or x86-64 instruction set architecture. But there are also other operating systems running on such processors, it doesn't have to be Windows.

Talking about a 64-bit Windows, you can see Win64 and AMD64 and x86-64 (sometimes even x64) as more or less equal. As long as it does not explicitly say IA-64 / Itanium.

[Atak_Snajpera]

I have double checked. Ripbot is using the same from it's temp directory. Maybe the commandline made by Ripbot is not ok for the encoder. I don't really know what is happening.

Maybe it has something to do with video decoding? I am using LSMASH.

Thanks for any kind of help.

x264 included in ripbot264 HAS support for OpenCL. Tested on crappy Geforce GT 710 and it works fine. Problem is with your drivers.

[mparade]

Thank you. I am using latest NVIDIA Game Ready Driver.

[sneaker_ger]

@mparade
Are you using 10 bit encoding? It seems x264 doesn't support opencl+10 bit and then throws this confusing warning about "not compiled with OpenCL support".

[mparade]

1 minute ago I realized it too. :). Thank you very much! In 8-bit it was working like a charm. I have to think over then if I really want 10bit encoding. All of my players support 10bit files, a bit slower encoding times but with a 2990WX...:)

[mparade]

Is it possible to have jobs deleted automatically after they are completed successfully? (Not in batch mode)

Thank you!

[Atak_Snajpera]

Sure.

[mparade]

May I ask where? In batch window I have found this option but nowhere else, please help.

Thank you in advance.

Edit: just found it...sorry for the question

[mparade]

Sometimes I have problems with dtsma sources.
Sometimes they recognized with faulty channel numbers, sometimes they are demuxed and after being chosen from the list relevant parameters and even profiles are not loaded at all. So I have to ignore them as a source stream. Is there a solution for this yet?

[SKPN]

Well, as I said, I'd like to have my 4K's at a "standard" of 60Mps, and again, as I've said, some have lower than that, some are higher, so for that I need to run them thru RB, even if it's video only.

This part makes no sense. You shouldn't be aiming for a specific bit rate unless you have a limited amount of space, and even then, should only mean lowering the bit rate. Encoding at a higher bit rate than the source video won't increase the quality, so there is no point in doing so; all you're doing is wasting space.

One thing I would like to get around to doing is upscaling some 1080p to 4K, but do any filtering, etc, still @ 1080p, then once satisfied, do a 4K HEVC encode, and see how it turns out.

This is not a good idea. The more you encode a video, the worse the quality will be. So if you take the 1080p source, apply filters and encode it, then take the encoded video and encode it again at 4K, it will look worse than doing both the upscale and filters in a single encode. Every time you re-encode a video, you're losing data.

[guest]

I noticed another small auto update today (just core.exe). A change to the command script, cmd features/control.

A lot of time seems to be getting spent on this script for when a job is completed...I for one don't really get the idea behind this, I think someone requested a feature for deleting completed jobs.....

If that's basically what it's for, I would prefer to do it manually !!

And what of my simple feature request, on RB gui's main window, the current core version/build is displayed (for quick reference)

[guest]

I will dig out a "crappy ssd", and try a "big ass 80Gb" encode, with the latest build of RB, and see just how long it does take....

If it still takes long time then I will rethink your request. Just give me some hard numbers.

Just a little update on my "big ass" encode....

I was only about an hour from it starting the muxing process, and the SSD I was using for the Temp files, ran out of space.

It was "only" 120Gb, and with the ONE movie to process, filled it up :(

So will have to re-think the setup, and start all over again :(

Regardless of that problem, I noticed that earlier on during the chunks being processed, that randomly a chunk would just "reset", and start again from 0%, so all it had processed up to that point, was lost. :(

So I turned off the re connect option, and it seemed to be OK, so something in the workings of that, seems to create a problem, unless there's a brief hiccup in the network connection.

[guest]

Would it be too much to ask if you could implement an option where you could send a shut down/kill command thru to an individual server's IP address, when ever you wanted or needed to ??

Preferably within the Encoding Client window/screen.

Very handy when the servers are elsewhere !!

Separate to the shut down options at the end of an encode.

[byteshare]

Well, as I said, I'd like to have my 4K's at a "standard" of 60Mps, and again, as I've said, some have lower than that, some are higher, so for that I need to run them thru RB, even if it's video only.
For what I want & need to do to audio, can be dealt with other app's, as too, the subtitles.
I have to agree, I don't like grain either, but fortunately most 4K's aren't very "noisy", so not a lot of filtering required.
I know I need to try a few different options, but it's the time factor that's my main concern...imagine doing a high bitrate 4K HEVC, on a Slow preset, using MDGrain2....day's :( even with my "farm".
But the other time waster is between when the encode is completed, and the start of the next job in queue...meaning the muxing, etc...it leaves a lot of resources doing nothing. :(, but having said that, I am yet to test this again, after putting some SSD's in the mix.
One thing I would like to get around to doing is upscaling some 1080p to 4K, but do any filtering, etc, still @ 1080p, then once satisfied, do a 4K HEVC encode, and see how it turns out.
But just imagine the time & resources involved when 8K is a lot more common place.
You'd want to look at some of the advanced x265/HEVC commands to preserve detail but you should still be able to reduce size while it being near same quality.
Personally I go for looks 95% precieved quality, so I don't have any good settings for you to start with.
That said, if you are using filters like MDGrain2, IMO you shouldn't worry about the bitrate because you're going to be chopping out a lot with that so you should just use CRF12 or so from what it I'm guessing you like. If you are using constant bitrate with MDegrain2 and it is set higher/same as source you're wasting bits.
If you're going to use constant bitrate without filters I'd set it a little lower than source at least and do 2pass and you'll likely get something 99.9999% the same if you can even see a difference (depending on your settings). Some HEVC settings do bluring, slight denoise/dehalo type stuff so if it isn't looking close to the source your settings are the issue.
From the sounds of it you need a bigger SSD, guessing at least 500GB or more.
As for upscaling the way you want to do it isn't a problem since you'd do the filters either before or after the upscale and upscale all in one job, then encode.

I noticed another small auto update today (just core.exe). A change to the command script, cmd features/control.
A lot of time seems to be getting spent on this script for when a job is completed...I for one don't really get the idea behind this, I think someone requested a feature for deleting completed jobs.....
If that's basically what it's for, I would prefer to do it manually !!
And what of my simple feature request, on RB gui's main window, the current core version/build is displayed (for quick reference)
I just got a chance to use the new update and it is great for me. Really reduced my manual tasks.
You can do a lot with it besides deleting source files.
Also, made sure when a lot of jobs fail my drives won't fill up.

Is it possible to have jobs deleted automatically after they are completed successfully? (Not in batch mode)
Settings > [uncheck] Keep jobs after successful conversion

[Dhry]

Okay, so today Ripbot updated core (2019.10.15), tried an encode of an MP4 file via adding to batch, but all it does is show "Waiting for file..." in the main window and nothing happens after that. Usually this is the point at which it demuxes etc. I tried it on multiple mp4s, always the same. Looking at the temp folder, the job1 folder is created but no work files are added to it.

[Atak_Snajpera]

Okay, so today Ripbot updated core (2019.10.15), tried an encode of an MP4 file via adding to batch, but all it does is show "Waiting for file..." in the main window and nothing happens after that. Usually this is the point at which it demuxes etc. I tried it on multiple mp4s, always the same. Looking at the temp folder, the job1 folder is created but no work files are added to it.
Waiting for file... means in practice waiting for file to be accessible. Are you sure nothing is blocking your files? I've just tested on my machine and everything works fine. Besides I didn't touch anything in code regarding this issue.

[Dhry]

Waiting for file... means in practice waiting for file to be accessible. Are you sure nothing is blocking your files? I've just tested on my machine and everything works fine. Besides I didn't touch anything in code regarding this issue.

Definitely not. I thought the same thing as well, and used Unlocker on each file to make certain that something had not blocked access to them. They were not opened for reading or writing by anything. I'll reboot my machine in a little while and will try again. Cheers!

[byteshare]

Definitely not. I thought the same thing as well, and used Unlocker on each file to make certain that something had not blocked access to them. They were not opened for reading or writing by anything. I'll reboot my machine in a little while and will try again. Cheers!
Did you make the job after the update, or before?
If before try deleting the job and remaking it.
After the reboot, if you're still having issues try posting the media info.

[Dhry]

Did you make the job after the update, or before?
If before try deleting the job and remaking it.
After the reboot, if you're still having issues try posting the media info.

Job was made after the update, with the shared job folder completely and absolutely empty. Deleted job and remade it, same issue. Deleted job, deleted job1 folder in temp folder, created again, same. Ripbot has worked in this configuration through multiple updates, but this last update is broken somehow. I started a batch encode and added a single file, this time from a completely different folder, absolutely nothing locking the file. Even if I click Abort Job Creation, it tells me "please wait.. aborting" and that never ends. I'm using a test mp4 file from one of my video cameras. Media info is as follows, hope this is the format you're asking for:

General
Complete name : K:\test.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 28.7 MiB
Duration : 39 s 0 ms
Overall bit rate mode : Variable
Overall bit rate : 6 166 kb/s
Encoded date : UTC 2018-08-04 00:02:07
Tagged date : UTC 2018-08-04 00:02:07

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 1 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 1 frame
Format settings, GOP : M=1, N=30
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 39 s 0 ms
Bit rate mode : Variable
Bit rate : 6 162 kb/s
Maximum bit rate : 30.7 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 30.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.099
Stream size : 28.6 MiB (100%)
Encoded date : UTC 2018-08-04 00:02:07
Tagged date : UTC 2018-08-04 00:02:07
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC

Screenshot of my batch settings attached.

EDIT: I added this file as normal (ie not through the Batch mode) and it worked. Normal add probes, demuxes, indexes etc as normal. Distributed encoding works great. Problem seems to be within the batch functionality.

[guest]

If it still takes long time then I will rethink your request. Just give me some hard numbers.

OK, here we go, this is what I observed, when encoding my "big ass" 4K, 60Mb/s bitrate encode.

Now I know a lot here cannot understand why I'm doing this the way I'm doing, (but that's my choice) so with that aside, what I have documented could impact nearly anyone that is doing 4K HEVC encodes, using Distributed Encoding.

So first things, I had an approx 65Gb, @ approx 65Mb/s bitrate movie (John Wick 3).

I queued it up, the initial demuxing & indexing of the file, took 27 minutes, using Lsmash.

Then the next day, when I started the job, it took a further 15 minutes, before it started encoding, and initiating the DE Servers, and I am ONLY doing the video.

So over the next 3 days (3 different encoding session's), probably turned out to be around 12 - 14 hour's using varying different server's in the "encoding farm".

Just a foot note, I have a lot of Solar Panels on the roof, and I won't do any long term processing jobs unless I can get it for "free", so that can change from day to day.

So, finally, late today, the chunks were all done.

So I got the phone out, and started the stop watch, and the combining of the chunks took 10 minutes 30, then the muxing process took a further 15 minutes 30, using SSD's for the Temp folder & where the encode is compiled.

I would suggest that this time would be shorter if the bitrate wasn't as high as I had it (1 Pass @ 60,000kb/s), ended up being 59.6Mb/s (Mediainfo).

I am yet to mux back in the audio (which I have changed from TrueHD to DTS-MA), and a subtitle track (.srt)

So, getting back to a question raised a week or so ago, to have RB to start the next job in the queue, whilst it is combining the chunks, and the final muxing. (Is muxing a single threaded process ?)

So observations of "my" process, there is approx 45 minutes where NONE of the DE Server's are being used, when starting a new 4K job, from scratch....then of course the DE Server's are going flat out 'til all the chunks have been completed.

Then, hopefully the Client PC is the one left doing the final combining & muxing, which leaves ALL the DE Servers doing nothing for a further 25 or so minutes. So they could be turned off until the next job has actually start encoding, for just over an hour. (Between jobs)

So as great as DE is, there can be a LOT of time that the servers aren't doing anything..so if there was a way to have RB to at least start the next job, once the chunks of the previous job were completed, that could save a reasonable amount of time, and power.

OR,

if there was someway for RB to start the DE servers once the job is about to start encoding, (when the chunks are being created) and maybe shutdown the DE servers once the chunks are done, instead of once the full encode was completed.

OR,

some option to manually turn the remote DE servers on & off from the Client pc.

[Atak_Snajpera]

So I got the phone out, and started the stop watch, and the combining of the chunks took 10 minutes 30, then the muxing process took a further 15 minutes 30, using SSD's for the Temp folder & where the encode is compiled.
~100 MiB/s copy speed on SSD<->SSD? Are you using TLC SSD? I'm asking because direct write to TLC NAND is around that level. QLC is even worse (~75MiB/s)

[guest]

~100 MiB/s copy speed on SSD<->SSD? Are you using TLC SSD? I'm asking because direct write to TLC NAND is around that level. QLC is even worse (~75MiB/s)

WD Greens in Raid 0 for Temp, some Chinese SSD for the Encodes.

That's all I had, spare.

[Ryushin]

Why not just use MakeMKV. Make a MKV with just the video and the audio you want. Then extract the subtitles out, OCR and convert them to SRT/ASS/etc, mux them back in, and your done. Your video quality will be exactly the same as the source. Most smaller than 60Mb/s and some a bit larger. Your finished file will done quicker than ever having to encode. From what I understand, unless you need to degrain, there is no need to re-encode the source.

When I encode, I use CRF18 for x265 and I cannot tell the difference between the original source and my output. Some 4K sources that have a huge amount of grain such as Blade Runner, come out looking far better than the original when using MDegrain2/3.

[Atak_Snajpera]

WD Greens in Raid 0 for Temp, some Chinese SSD for the Encodes.

That's all I had, spare.

Could you test your SSD/HDD in my benchmark tool?
http://www.mediafire.com/file/i630b6n1fal38fj/IOSpeedTester.7z/file

Results on my 5 years old 120GB SSD (MLC)
https://i.imgsafe.org/65/652bfb9795.png

It is a command line tool. Use these switches

IOSpeedTester.exe --test-location "X:\" --transfer-size 120GIB --log-file "C:\users\%username%\Desktop\MyLog.csv"


Where X is a drive letter of your drive.

Upload MyLog.csv somewhere.

full help
IOSpeedTester v1.0 by Atak_Snajpera
Syntax: IOSpeedTester [options]

Options:
--help Show this help text and exit
--test-location <string> Folder where test files will be stored
--threads <1..64> Number of working threads. [1]
--buffer-size <1KiB..64MiB> Write/Read buffer size. [16MiB]
--transfer-size <integer> Amount of bytes to transfer. [1GiB]
--buffered-io Enables IO buffering in RAM. [Unbuffered]
--log-file <string> Log information to specific file. [None]
--write-only Performs write test only. [Write+Read]

Example usage:

Sequential Test with logging to a file
IOSpeedTester --test-location "%temp%" --log-file "MyLog.csv"

MIN IOPS Test
IOSpeedTester --test-location "%temp%" --buffer-size 4KiB

MAX IOPS Test
IOSpeedTester --test-location "%temp%" --buffer-size 4KiB --threads 64

[byteshare]

EDIT: I added this file as normal (ie not through the Batch mode) and it worked. Normal add probes, demuxes, indexes etc as normal. Distributed encoding works great. Problem seems to be within the batch functionality.
Batch function is working for me on 2 different systems...I'm not sure what is happening for you. I'll try to think on it.

[byteshare]

I was trying to encode some HDR and I normally don't define the colorprim, transfer, or colormatrix. I also don't use the tone mapping.
I'm getting the error: x265 [error]: invalid argument: transfer = bt.2020
With tone mapping I don't, as I'd expect.
I tried defining the colorprim, transfer, and colormatrix:
--colorprim bt2020 --transfer bt2020-12 --colormatrix bt2020c
Because it looked like RB was doing it wrong (media info at the bottom), with instead:
--colorprim bt2020 --transfer bt.2020 constant --colormatrix bt2020c
But I'm seeing both RB and my settings in the the Encoding Client:
Encoding started...
""\\SERVER\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\SERVER\RipBot264temp\job36\Chunks\1.avs" -strict -1 -f yuv4mpegpipe -
| "\\SERVER\Ripbot264temp\tools\x265\x265_x64.exe" --seek 0 --colorprim bt2020 --transfer bt.2020 constant --colormatrix bt2020c --crf 22 --fps
24000/1001 --min-keyint 24 --keyint 240 --frames 1405 --sar 1:1 --profile main10 --output-depth 10 --colorprim bt2020 --transfer bt2020-12
--colormatrix bt2020c --aq-mode 3 --ctu 64 --y4m --pools "+" --output "\\SERVER\RipBot264temp\job36\Chunks\1.265" -"
x265 [error]: invalid argument: transfer = bt.2020

Is this a RB thing or what am I doing wrong?

Media Info:
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L6@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 48 min 21 s
Bit rate : 64.5 Mb/s
Width : 4 096 pixels
Height : 2 304 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.285
Stream size : 21.8 GiB (100%)
Writing library : x265 2.4+27-e9e574bbed93:[Windows][GCC 6.3.0][64 bit] 10bit
Encoding settings : cpuid=1173503 / frame-threads=2 / numa-pools=4 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 /
input-csp=1 / input-res=4096x2304 / interlace=0 / total-frames=69568 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance
/ no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / bframes=8 /
b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect
/ no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip
/ nr-intra=0 / nr-inter=0 / no-constrained-intra / no-strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 /
merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=-3:-3 / no-sao / no-sao-non-deblock / rd=4 / no-early-skip
/ rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / rdpenalty=0 / psy-rd=0.30 / psy-rdoq=2.50 / no-rd-refine / analysis-mode=0
/ no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=16.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30
/ aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / sar=0 / overscan=0
/ videoformat=5 / range=0 / colorprim=9 / transfer=15 / colormatrix=10 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 /
max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps
/ scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / no-hdr-opt / no-dhdr10-opt / refine-level=5 / no-limit-sao / ctu-info=0
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : BT.2020 (12-bit)
Matrix coefficients : BT.2020 constant

(added linebreaks for scrolling reduction)

[Atak_Snajpera]

Send me 100MiB sample with that BT.2020 (12-bit) transfer characteristics.

[slalom]

So as great as DE is, there can be a LOT of time that the servers aren't doing anything..so if there was a way to have RB to at least start the next job, once the chunks of the previous job were completed, that could save a reasonable amount of time, and power.
But Temp disk is being used for the previous job, you will slow it down if the program starts a new job simultaneously

[byteshare]

But Temp disk is being used for the previous job, you will slow it down if the program starts a new job simultaneously
Good point. If you're muxing on the temp drive while trying to transfer the video to the job folder at the same time that would only compound things.
You'd have to pre-load the video for the next job once the current job starts to get around that but still not starting while the last job is muxing...tricky timing.

[guest]

Good point. If you're muxing on the temp drive while trying to transfer the video to the job folder at the same time that would only compound things.
You'd have to pre-load the video for the next job once the current job starts to get around that but still not starting while the last job is muxing...tricky timing.

You're probably correct, but at least while the muxing is being done, the next job is at least starting it's process, which could save some time & resource's.

Even if the muxing & copying was a little slower, you're not having to wait for one job to be completed.

But here's an idea, what if there was a second temp drive, and you allocated which temp drive to be used, when creating new jobs. (wouldn't work too well with batching)

[byteshare]

You're probably correct, but at least while the muxing is being done, the next job is at least starting it's process, which could save some time & resource's.
Even if the muxing & copying was a little slower, you're not having to wait for one job to be completed.
That is kind of the point, it would slow it down enough that your next job would actually be encoding later than the way it works now. Yes, technically the job would "start" sooner but the time for the video to copy to the temp drive would take longer with the last job muxing at the same time.
The only way it would be "faster" is if once jobs are started the next job loads the video while the other job is still encoding, then muxing would happen and the next job would start, then while that job is still going but after the last muxing is done the next-next job would start copying the video...this only really works if your jobs are both small enough/slow enough encodes that the timing can work...like I was saying, it would be tricky and wouldn't be helpful in many cases.
...Ideally if some symbolic link to the original file could be used for the sharing (in DE) so that you wouldn't have to even copy a file, then it would be much simpler/faster to start the next job while the last job is muxing.
But here's an idea, what if there was a second temp drive, and you allocated which temp drive to be used, when creating new jobs. (wouldn't work too well with batching)
Wouldn't work for a lot of people I'd guess. Wouldn't work for me. I'd have to get a 3rd drive.

[guest]

That is kind of the point, it would slow it down enough that your next job would actually be encoding later than the way it works now. Yes, technically the job would "start" sooner but the time for the video to copy to the temp drive would take longer with the last job muxing at the same time.
The only way it would be "faster" is if once jobs are started the next job loads the video while the other job is still encoding, then muxing would happen and the next job would start, then while that job is still going but after the last muxing is done the next-next job would start copying the video...this only really works if your jobs are both small enough/slow enough encodes that the timing can work...like I was saying, it would be tricky and wouldn't be helpful in many cases.
...Ideally if some symbolic link to the original file could be used for the sharing (in DE) so that you wouldn't have to even copy a file, then it would be much simpler/faster to start the next job while the last job is muxing.

Good to see you've given this some thought, as well.

So I have to ask, do you do many, if any, 4K movie encodes, and how many pc's can you use for DE ??

I'm just trying to figure out some way to utilise the DE servers more energy efficiently. It just frustrates me to see several servers whirring away, using power, but not doing any encoding :(

Wouldn't work for a lot of people I'd guess. Wouldn't work for me. I'd have to get a 3rd drive.

Well, if I step up to using SDD's or NVMe's for the temp & encodes drives, I'm going to have to buy a LOT :(

But if a 2nd temp drive could be implemented, I think it would be worth having to buy another drive :)

[guest]

Could you test your SSD/HDD in my benchmark tool?
http://www.mediafire.com/file/i630b6n1fal38fj/IOSpeedTester.7z/file

Upload MyLog.csv somewhere.

OK, have done your test's as requested, I hope I have provided the info you wanted to know.

I checked ALL the drives in that PC, just for comparison, some didn't have enough free space to do the 120Gb test, but the speeds are still relevant, I'd say.

Now remembering that these SSD's aren't anything special, they were just a few I had lying around.

http://www.mediafire.com/file/dqlf6cqxspncodu/Desktop.rar/file

[byteshare]

Good to see you've given this some thought, as well.

So I have to ask, do you do many, if any, 4K movie encodes, and how many pc's can you use for DE ??

I'm just trying to figure out some way to utilise the DE servers more energy efficiently. It just frustrates me to see several servers whirring away, using power, but not doing any encoding :(
I do some 4K. I've been doing more and more with time.
I have had up to 6 computers in DE mode, but right now I use 2 since upgrading some systems (made the others ones pointless), and 1-2 other computers with a local only DE mode. I see no reason you can't easily have
8-16 computers being used on one jobs (assuming longer/high CPU jobs).
The main thing is having a fast temp drive for large files.
Well, if I step up to using SDD's or NVMe's for the temp & encodes drives, I'm going to have to buy a LOT :(

But if a 2nd temp drive could be implemented, I think it would be worth having to buy another drive :)

The main thing would be using the SSD/M.2 for the temp drive (locally mux) then if you still want it somewhere else, have it auto-move to another drive with the new after job scripting to allow the next job will start while it is moving.

[Atak_Snajpera]

OK, have done your test's as requested, I hope I have provided the info you wanted to know.

I checked ALL the drives in that PC, just for comparison, some didn't have enough free space to do the 120Gb test, but the speeds are still relevant, I'd say.

Now remembering that these SSD's aren't anything special, they were just a few I had lying around.

http://www.mediafire.com/file/dqlf6cqxspncodu/Desktop.rar/file

Your SSDs are even worse than your HDDs! :eek:
No wonder that muxing takes ages! You really need some good MLC SSD instead of those TLC garbage.

https://i.ibb.co/z4vWySb/ssd-samsung-evo-840.png

https://i.ibb.co/5WSSQW0/SSD-Veseky.png

https://i.ibb.co/D4wCVfc/hdd-Seagate.png

https://i.ibb.co/vVV6Ty7/hdd-Wd-green.png

[guest]

Your SSDs are even worse than your HDDs! :eek:
No wonder that muxing takes ages! You really need some good MLC SSD instead of those TLC garbage

Not that surprised by that...I did say they were all I had...

So, what would you suggest...SSD's or PCI-e mounted NVMe's ??

[byteshare]

Your SSDs are even worse than your HDDs! :eek:
No wonder that muxing takes ages! You really need some good MLC SSD instead of those TLC garbage.
Wow, I've never seen that. Maybe they're also near the end of their life?
TY for the graphs.

So, what would you suggest...SSD's or PCI-e mounted NVMe's ??
Those or M.2. Pretty much any drive that does 500MB/s or more read/write, which is pretty common these days.
I have an M.2 that does around 2GB/s for read/write.

[guest]

Wow, I've never seen that. Maybe they're also near the end of their life?
TY for the graphs.


Those or M.2. Pretty much any drive that does 500MB/s or more read/write, which is pretty common these days.
I have an M.2 that does around 2GB/s for read/write.

Yeah, pretty bad :(

Thanks, I'll get the best I can that will suit my MB (no native support)

[Atak_Snajpera]

Wow, I've never seen that. Maybe they're also near the end of their life?
TY for the graphs.
No, it is because SLC cache is tiny (around 3 GiB)

So, what would you suggest...SSD's or PCI-e mounted NVMe's ??
Obviously something using NVMe. Downside of PCI-e is that you most likely won't be able to boot from that drive. Before you buy anything make sure that write speed is good across whole capacity. Do not be fooled by that SLC-cache used in TLC/QLC drives.

[mparade]

1) Source files are on your home PC.
2) Work PC just needs EncodingServer.exe running!
3) The idea is that you connect via VPN to work PC and then Work PC sees shared folder (for example \\HOME-PC\RipBot264temp) on Home PC.

With VPN connection established using my remote PC, servers of the remote PC remain in offline mode even after trying to connect to them from home PC (after pressing ON button it tries to connect to remote servers but after some second it returns to offline mode again). Should I set something else as well? My admin told VPN now should be working. Please help. Thank you.

[guest]

I have an M.2 that does around 2GB/s for read/write.

Hey, well, I think I've decided what I want to try, and that is get a couple of NVMe PCI-e Adapter's, and maybe 500Gb M.2 Samsung 970's (or similar).

Might as well spend a little more, than "settle" for some "fast" SSD's, they are so 2017 :rolleyes:

And to a point that Atak made, PCI-e mounted NVMe's may not boot, but I won't be using them as the boot / OS drive.

[guest]

Your SSDs are even worse than your HDDs! :eek:

https://i.ibb.co/vVV6Ty7/hdd-Wd-green.png

I just noticed that these are depicted as being Green HDD's, but indeed, they were Green SSD's in a RAID 0...

So that show's you how piss weak they are !!!

[byteshare]

With VPN connection established using my remote PC, servers of the remote PC remain in offline mode even after trying to connect to them from home PC (after pressing ON button it tries to connect to remote servers but after some second it returns to offline mode again). Should I set something else as well? My admin told VPN now should be working. Please help. Thank you.
Can you reach the share manually from the remote PC?
Can you ping the remote PC?

[guest]

Obviously something using NVMe. Downside of PCI-e is that you most likely won't be able to boot from that drive. Before you buy anything make sure that write speed is good across whole capacity. Do not be fooled by that SLC-cache used in TLC/QLC drives.

Could you please give me your honest opinion on these items ??

NVMe PCI-e adaptors

https://www.amazon.com/Silverstone-SST-ECM24-SuperSpeed-PCI-heatsink/dp/B07MG73P8P/ref=sr_1_7?keywords=Silverstone+M.2+NVMe+SSD+NGFF&qid=1571467552&sr=8-7

or

https://www.amazon.com/SilverStone-Technology-Adapter-Integrated-Heatsink/dp/B07MRKNH6G/ref=sr_1_3?keywords=SilverStone+ECM23+M.2+to+PCIe+x4&qid=1571467714&sr=8-3

500Gb NVMe's

https://www.amazon.com/XPG-SX8200-Gen3x4-3000MB-ASX8200PNP-512GT-C/dp/B07K1HMMJC/ref=sr_1_1?keywords=ADATA+XPG+512GB+SX8200+Pro&qid=1571467837&sr=8-1

or

https://www.amazon.com/Samsung-970-EVO-500GB-MZ-V7E500BW/dp/B07BN4NJ2J/ref=sr_1_2?crid=2VNB3BDWFVT5W&keywords=samsung+970+nvme+500gb&qid=1571467930&sprefix=Samsung+970+nvme%2Caps%2C387&sr=8-2

At this stage, I think I would go a 500Gb for the temp drive, and 250Gb for the encode drive, and just "clean up" after every job or 2.

[Atak_Snajpera]

500Gb NVMe's

https://www.amazon.com/XPG-SX8200-Ge...1467837&sr=8-1

or

https://www.amazon.com/Samsung-970-E...s%2C387&sr=8-2

https://cdn.mos.cms.futurecdn.net/rJVbF7mbkoW3awfWyKp8Bo-650-80.png

Like I said, if you can afford buy Samsung 970 PRO (MLC)... Adata sx8200 also looks good until 60GiB mark.

[Atak_Snajpera]

With VPN connection established using my remote PC, servers of the remote PC remain in offline mode even after trying to connect to them from home PC (after pressing ON button it tries to connect to remote servers but after some second it returns to offline mode again). Should I set something else as well? My admin told VPN now should be working. Please help. Thank you.

1) run encodingserver.exe on remote pc and take a screenshot.
2) run ripbot264.exe on your home pc. go to settings->Distributed encoding and show me what client ips are listed under CLIENT IP
3) Are you sure firewall on remote pc does block connection?

[guest]

https://cdn.mos.cms.futurecdn.net/rJVbF7mbkoW3awfWyKp8Bo-650-80.png

Like I said, if you can afford buy Samsung 970 PRO (MLC)... Adata sx8200 also looks good until 60GiB mark.

I would be interested to see this info for the new'ish 970 Plus.

Found some :- not too good :(

https://www.tomshardware.com/reviews/samsung-970-evo-plus-ssd,5608-2.html

Adata is 2nd best, but a fair bit off the mark :(

I think the Pro is a little too exy for me, the Plus is substantially cheaper, but might be only a little "slower"

And are those adapters suitable ??

[Atak_Snajpera]

24 GiB file copy test
https://pclab.pl/zdjecia/artykuly/wkoczyk/201901/evo_plus/kop_bduzy_z.png

And are those adapters suitable ??
yes

[guest]

24 GiB file copy test

:thanks:

Interesting, but I'm now undecided, I don't want to get 2 x 512Gb Pro's, meaning one for Temp, one for Encoding....which would benefit more with a Pro ?, as compared to say the Plus, or the Adata XPG ??

[Atak_Snajpera]

Adata xpg has very good write speed up to 60GiB so it should fit to your needs. You should be able to combine chunks and mux in less than 2 minutes. Still huge improvement from your old ssds with ~100MiB/s write speed.

[guest]

Adata xpg has very good write speed up to 60GiB so it should fit to your needs. You should be able to combine chunks and mux in less than 2 minutes. Still huge improvement from your old ssds with ~100MiB/s write speed.

I think your crystal ball is working again, 'cause that's exactly what I was thinking...if it's good for 60Gb, that is about the size of the final encode. :)

OK, so most of these posts have been about getting the combining of chunks, and the muxing process done a lot faster...this should also affect the initial copying & indexing of the job, correct ??

And one other thought about the 970 Pro...would it be another option to get a 1tb "stick", and then partition it, in say a 60/40 split....I know it's 2 drives on the same "stick", but with the performance of it, it should be able to still read & write fast, at the same time.

[Atak_Snajpera]

Indexing will be most likely slowed down by performance of your CPU (single core). Remuxing is basicaly a copy process.

[skylinekiller]

So it appears the "Default.style" file is just for decoration. I just downloaded, updated and reinstalled the latest RipBot. No matter what numbers I change, the font is always the same. Where can i change the default subtitle characteristics? The RipBot264\Tools\AviSynth plugins\VSFilter seems to be a dead-end.

Is this something that can be remedied in the next update?

[Atak_Snajpera]

So it appears the "Default.style" file is just for decoration. I just downloaded, updated and reinstalled the latest RipBot. No matter what numbers I change, the font is always the same. Where can i change the default subtitle characteristics? The RipBot264\Tools\AviSynth plugins\VSFilter seems to be a dead-end.

Is this something that can be remedied in the next update?

Latest version of VSfilter indeed stopped reading .style file for .srt subtitles. You will have to always convert .srt subtitles to .ass/.ssa and disable shadow directly. I could automate this process if this really annoys you.

[skylinekiller]

You will have to always convert .srt subtitles to .ass/.ssa and disable shadow directly. I could automate this process if this really annoys you.

That what I have been doing since it broke. It does take away the luxury of it being automated. It would bee nice if you could automate the process with the attached settings in a future update. I'm sure there are more pressing things to fix, but I do a lot of foreign films with subtitles.

Thank you

[guest]

And what of my simple feature request, on RB gui's main window, the current core version/build is displayed (for quick reference)

I thought you could do something like this :-

http://www.mediafire.com/file/o6ktod3uk23tkd5/RB.jpg/file

There must be someway to "link" it to the info in the Tool's list, so that it updates itself.

[slalom]

dtsma track is recognised as dts in the audio section of the job

dtsma track was recognised as dtsma in the past

[Atak_Snajpera]

dtsma track is recognised as dts in the audio section of the job

dtsma track was recognised as dtsma in the past

Bug in mediainfo
https://forum.doom9.org/showthread.php?p=1887964#post1887964

[byteshare]

Send me 100MiB sample with that BT.2020 (12-bit) transfer characteristics.
Not sure what you changed but is working with Core 2019.10.19 Thank you!

[Atak_Snajpera]

That what I have been doing since it broke. It does take away the luxury of it being automated. It would bee nice if you could automate the process with the attached settings in a future update. I'm sure there are more pressing things to fix, but I do a lot of foreign films with subtitles.

Thank you

Done. Edit default.style in \Tools\SRTtoSSA folder

[guest]

Done. Edit default.style in \Tools\SRTtoSSA folder

Got the latest update this morning, but something went wrong (tried it twice)

2019-10-23 10:00:25 : New section [SRTtoSSA] detected
2019-10-23 10:00:25 : Downloading file http://atak-snajpera.5v.pl/ripbot264update/SRTtoSSA.zip to D:\RipBot264v1.25.0\Updates\SRTtoSSA.zip
2019-10-23 10:00:25 : [ERROR] File http://atak-snajpera.5v.pl/ripbot264update/SRTtoSSA.zip could not be downloaded!
2019-10-23 10:00:25 : Downloading finished.

And how do you access what this does ??

Such a shame that there isn't an easy .sub to .srt conversion :(

[ReinerSchweinlin]

Just doing some encodes on some cartoon and anime stuff. While previewing, I noticed many duplicate frames. Some time ago, I used Donads DUP Filter to optimize this - so I tried to use it in Ripbot... But it´s too old, 32Bit, etc...
I could use some help implementing a filter like the old DUP. Any hints anyone?

[byteshare]

Such a shame that there isn't an easy .sub to .srt conversion :(
Yeah, that is because .subs are imaged based so it isn't as simple to translate them.

Just doing some encodes on some cartoon and anime stuff. While previewing, I noticed many duplicate frames. Some time ago, I used Donads DUP Filter to optimize this - so I tried to use it in Ripbot... But it´s too old, 32Bit, etc...
I could use some help implementing a filter like the old DUP. Any hints anyone?
Either you'll need to find an updated or alternative filter for 64B support or use an old version of RB :(
With anime it is pretty normal to have duplicate frames without variable frame rate. Using x265 I don't find it is a major impact on file size or encoding time.

[guest]

Yeah, that is because .subs are imaged based so it isn't as simple to translate them.

Yep, thank goodness for SubtitleEdit :cool:

[skylinekiller]

Done. Edit default.style in \Tools\SRTtoSSA folder

Thank you very much, but I don't see a default file in that folder. Do I copy the one from VSFilter folder and use it in the SRTtoSSA folder?

[guest]

Thank you very much, but I don't see a default file in that folder. Do I copy the one from VSFilter folder and use it in the SRTtoSSA folder?

Are you sure you have that folder ??

I got the latest update earlier today, and there was a problem downloading that item.

There's an ssatosrt folder, but not the srttossa.. :(

[sneaker_ger]

Latest version of VSfilter indeed stopped reading .style file for .srt subtitles.
I tried the vsfilter from the package in the start post. It still works. Make sure the .style file ends with a newline.

[byteshare]

Would it be possible in a future update to change some of the EncodingClient behavior so that it can stay minimized (if you minimize it) or not steal focus when a new job comes up?
This isn't urgent but it was less noticeable for me before some of the fixes that sped up my encodes. While working on other things, the EncodingClient is either popping up in-front of things and/or taking focus.
(similar to how the main RB window can stay minimized or the encoding servers)

[Atak_Snajpera]

Would it be possible in a future update to change some of the EncodingClient behavior so that it can stay minimized (if you minimize it) or not steal focus when a new job comes up?
This isn't urgent but it was less noticeable for me before some of the fixes that sped up my encodes. While working on other things, the EncodingClient is either popping up in-front of things and/or taking focus.
(similar to how the main RB window can stay minimized or the encoding servers)

If Ripbot264.exe is minimized then EncodingClient.exe will also start as minimized (focus is not stolen). Works fine on my win7.

[skylinekiller]

Are you sure you have that folder ??

I got the latest update earlier today, and there was a problem downloading that item.

There's an ssatosrt folder, but not the srttossa.. :(

Yes, please see the pic I posted. That is the folder with the files inside of it. no Default file though

[guest]

Got the latest update this morning, but something went wrong (tried it twice)

2019-10-23 10:00:25 : New section [SRTtoSSA] detected
2019-10-23 10:00:25 : Downloading file http://atak-snajpera.5v.pl/ripbot264update/SRTtoSSA.zip to D:\RipBot264v1.25.0\Updates\SRTtoSSA.zip
2019-10-23 10:00:25 : [ERROR] File http://atak-snajpera.5v.pl/ripbot264update/SRTtoSSA.zip could not be downloaded!
2019-10-23 10:00:25 : Downloading finished.



And how do you access what this does ??



(edit), so another auto update today, and this time, this DID come in...

[guest]

Yes, please see the pic I posted. That is the folder with the files inside of it. no Default file though

Cannot open your attachment, you need to post them some other way.

See if you can get the very latest auto update !!

[byteshare]

If Ripbot264.exe is minimized then EncodingClient.exe will also start as minimized (focus is not stolen). Works fine on my win7.
Ah, okay. I usually keep RB (main window) up to see where in the queue I am but like the Client minimized. If I keep everything minimized it all stays minimized.

[guest]

Ah, okay. I usually keep RB (main window) up to see where in the queue I am but like the Client minimized. If I keep everything minimized it all stays minimized.

Are you really sure about that, byteshare ??

Mind you, doesn't the Client maximise, when it start's a new Job ?!?

I can have ANY combo I like open, either both minimized, both maximized, or either one, min. &/or max. (Windows 10 1903 19h1 & Windows Server 2019).

[guest]

That said, if you are using filters like MDGrain2, IMO you shouldn't worry about the bitrate because you're going to be chopping out a lot with that so you should just use CRF12 or so from what it I'm guessing you like.

Just wanted to say that I did a 1080p x264 to 1080p x265 conversion today, 10 bit Profile, Tune set to Grain, and using CRF 16 (didn't know how to get any lower).

It came out with a lower bitrate, approx 2Gb smaller than original, but it was pretty damn good, a lot more defined & sharper.

I am going to compare it to the same settings, BUT 1 pass @ 30Mp/s, and see if I can tell the difference.

[Atak_Snajpera]

(didn't know how to get any lower).
Use keyboard.

[brumsky]

@atak_snajpera

Could you please add an option for AAC 7.1?

thanks for the great software!

[byteshare]

Are you really sure about that, byteshare ??

Mind you, doesn't the Client maximise, when it start's a new Job ?!?

I can have ANY combo I like open, either both minimized, both maximized, or either one, min. &/or max. (Windows 10 1903 19h1 & Windows Server 2019).
Yes, on Win10, if I have both minimized they both stay minimized but if I have one open the client comes back when a new job starts.
Now that I realize the behavior I just keep the RB window minimized after looking at it.

[guest]

Use keyboard.

:thanks:

[guest]

Like I said, if you can afford buy Samsung 970 PRO (MLC)... Adata sx8200 also looks good until 60GiB mark.

OK, so I now have my 2 ADATA SX8200's installed, and I just did your IO tester on them..so I'm sure you're interested in the numbers, so could you kindly suss out the .csv's, and post the graphs & comment :)

https://www.mediafire.com/file/ifcdsdsm64g0jau/Desktop.rar/file

[Atak_Snajpera]

OK, so I now have my 2 ADATA SX8200's installed, and I just did your IO tester on them..so I'm sure you're interested in the numbers, so could you kindly suss out the .csv's, and post the graphs & comment :)

https://www.mediafire.com/file/ifcdsdsm64g0jau/Desktop.rar/file

I would return them ASAP. SLC cache only 5 GiB?!?

https://i.imgsafe.org/43/43532ed597.png

https://i.imgsafe.org/43/43532ddedc.png

Read speed looks good though...
https://i.imgsafe.org/43/433e25ebe7.png

[guest]

I would return them ASAP. SLC cache only 5 GiB?!?

WOW, that is EXTREMELY disappointing !!!, mind you, not too sure what that SLC cache means :(

I doubt I will return them, but I certainly wouldn't buy then again, but as they are WAY better than anything else I have.

I did a test on a RAID-0 of Seagate 3Tb 7200rpm spinners, and it was surprising fast...

So, with these being 512Mb NVME's, the performance must be down on the 1Tb that everyone seems to write about.

Could it be the adapter's...could it be the "slot" I have them in ??

The adapters I got are a full x16 type, and I have them in a x16 slot.

Would it be better or worse if they were in a x8 slot, even though I would think they would only use the speed they need.

[Atak_Snajpera]

WOW, that is EXTREMELY disappointing !!!, mind you, not too sure what that SLC cache means :(

I doubt I will return them, but I certainly wouldn't buy then again, but as they are WAY better than anything else I have.

I did a test on a RAID-0 of Seagate 3Tb 7200rpm spinners, and it was surprising fast...

So, with these being 512Mb NVME's, the performance must be down on the 1Tb that everyone seems to write about.

Could it be the adapter's...could it be the "slot" I have them in ??

The adapters I got are a full x16 type, and I have them in a x16 slot.

Would it be better or worse if they were in a x8 slot, even though I would think they would only use the speed they need.

You may try doing RAID-0 to at least get consistent 1 GiB/s write speed...

[guest]

You may try doing RAID-0 to at least get consistent 1 GiB/s write speed...

So you don't think it would change anything using different PCI-e slots ??

And the only way to create a RAID-0 would be in Windows Disk Management.

I will suss that out, seeing I haven't really used them yet, and I will send you the results.

BTW, were there different results in the 60Gb test ??

[Atak_Snajpera]

So you don't think it would change anything using different PCI-e slots ??

And the only way to create a RAID-0 would be in Windows Disk Management.

I will suss that out, seeing I haven't really used them yet, and I will send you the results.

BTW, were there different results in the 60Gb test ??

Read speed clearly shows that everything is ok with your setup
https://i.imgsafe.org/43/433e25ebe7.png

No. 60GiB test is basically a truncated 120GiB test. Write speed sucks because you bought TLC drive instead of MLC.

[guest]

Read speed clearly shows that everything is ok with your setup

No. 60GiB test is basically a truncated 120GiB test. Write speed sucks because you bought TLC drive instead of MLC.

OK, fair enough, thanks for this.

I guess I'll have to do a bit more "research", but I guess I should just "man up" and get the 970 Pro's that you suggested in the first place.

I will have to reach out to byteshare to test his NVMe's, and see what results he gets.

[Atak_Snajpera]

I guess I'll have to do a bit more "research", but I guess I should just "man up" and get the 970 Pro's that you suggested in the first place.
yeah. If your workflow requires high consistent write speed then TLC/QLC is a big no no.

[guest]

Could you test your SSD/HDD in my benchmark tool?
http://www.mediafire.com/file/i630b6n1fal38fj/IOSpeedTester.7z/file

Results on my 5 years old 120GB SSD (MLC)
https://i.imgsafe.org/65/652bfb9795.png

It is a command line tool. Use these switches

IOSpeedTester.exe --test-location "X:\" --transfer-size 120GIB --log-file "C:\users\%username%\Desktop\MyLog.csv"


Where X is a drive letter of your drive.

Upload MyLog.csv somewhere.

full help
IOSpeedTester v1.0 by Atak_Snajpera
Syntax: IOSpeedTester [options]

Options:
--help Show this help text and exit
--test-location <string> Folder where test files will be stored
--threads <1..64> Number of working threads. [1]
--buffer-size <1KiB..64MiB> Write/Read buffer size. [16MiB]
--transfer-size <integer> Amount of bytes to transfer. [1GiB]
--buffered-io Enables IO buffering in RAM. [Unbuffered]
--log-file <string> Log information to specific file. [None]
--write-only Performs write test only. [Write+Read]

Example usage:

Sequential Test with logging to a file
IOSpeedTester --test-location "%temp%" --log-file "MyLog.csv"

MIN IOPS Test
IOSpeedTester --test-location "%temp%" --buffer-size 4KiB

MAX IOPS Test
IOSpeedTester --test-location "%temp%" --buffer-size 4KiB --threads 64

Hey byteshare, would you be so kind as to download this, and runs the tests on your NVME's, and post of to Atak.

[guest]

yeah. If your workflow requires high consistent write speed then TLC/QLC is a big no no.

Well, this was all to get the final processes of RB to be faster.

But like I said, I will do the RAID-0 set up (which unfortunately will only leave me with 1 drive, instead of 2). So not sure what to use it for, for max. benefit (meaning either the Temp drive, or the drive that the encode is compiled to).

Anyway, I have a few setup options to try.

[byteshare]

So you don't think it would change anything using different PCI-e slots ??
Depends on your board, even if all of your PCIe ports are 16x, it doesn't mean they'll run at 16x when they're all in use, or even all at the same PCIe version. What is your mother board?

And the only way to create a RAID-0 would be in Windows Disk Management.
If you're using Windows 8x or higher you can also use Storage Spaces, which is built into Windows now.

Hey byteshare, would you be so kind as to download this, and runs the tests on your NVME's, and post of to Atak.
Samsung 970 EVO 500GB
Results: https://pastebin.com/LDwWrU10
Gets slower and slower over time while writing, but didn't drop below 700MiB/s for the 120GiB file. The read seemed more stable at 3GiB/s
@Atak_Snajpera what are you using to graph this?

[Atak_Snajpera]

https://i.imgsafe.org/48/48b959c7a9.png

https://i.imgsafe.org/48/48b959631e.png

@Atak_Snajpera what are you using to graph this?
LibreOffice Calc

@Pauly Dunne
Are you sure that you bought this model?
https://www.amazon.com/XPG-SX8200-Gen3x4-3000MB-ASX8200PNP-512GT-C/dp/B07K1HMMJC/ref=sr_1_5?crid=1HUJVI2W13NQE&keywords=xpg+8200+pro&qid=1572115673&sprefix=xpg+8200+pro%2Caps%2C253&sr=8-5

[guest]

@Pauly Dunne
Are you sure that you bought this model?
https://www.amazon.com/XPG-SX8200-Gen3x4-3000MB-ASX8200PNP-512GT-C/dp/B07K1HMMJC/ref=sr_1_5?crid=1HUJVI2W13NQE&keywords=xpg+8200+pro&qid=1572115673&sprefix=xpg+8200+pro%2Caps%2C253&sr=8-5

Yep, just checked with the packet it came in.

Something strange going on here, I think !!!

Also, thankyou byteshare :)

[Atak_Snajpera]

Yep, just checked with the packet it came in.

Something strange going on here, I think !!!

Also, thankyou byteshare :)

Yeah that advertised write speed of ~3GiB/s is impossible to achieve in real world scenarios.

BTW. This https://www.tweakguides.com/Hardcon19_7.html confirms that SLC cache for samsung 970 evo 500 GB is only 22GiB so my benchmark works correctly.

[Atak_Snajpera]

I think I know why they advertise that ssd as 3GiB/s+ ... They use ATTO benchmark with tiny file size (for example 256MiB)! This means that they are literally testing write/read on DRAM cache.
example
https://i.imgur.com/3tzNSfj.png

[guest]

Depends on your board, even if all of your PCIe ports are 16x, it doesn't mean they'll run at 16x when they're all in use, or even all at the same PCIe version. What is your mother board?

ASUS P9X79-E WS (a bit old I know), I've since tried them in various slots, and there's basically no difference.

If you're using Windows 8x or higher you can also use Storage Spaces, which is built into Windows now.

Tried to set up a Storage Space...didn't want to do it for me :( And a "Spanned" set up didn't make any noticable difference either.

Samsung 970 EVO 500GB
Results: https://pastebin.com/LDwWrU10
Gets slower and slower over time while writing, but didn't drop below 700MiB/s for the 120GiB file. The read seemed more stable at 3GiB/s
@Atak_Snajpera what are you using to graph this?

I guess I'll just ignore all this "testing", and see what (if any) difference it makes to the RB process...HAS to better than anything else I currently have.

Thanks for your input :)

PS:- I should have asked you if you're using PCI-e adapter's, or directly into the MB.

[guest]

I think I know why they advertise that ssd as 3GiB/s+ ... They use ATTO benchmark with tiny file size (for example 256MiB)! This means that they are literally testing write/read on DRAM cache.

So it's all sales bullshit, then...that sux !!!

Just out of interest I went looking for a Youtube clip of a test of the Adata drive, and this guy (who prefers Samsung's), said the Adata's were quite comparable, and he showed some Crystal Disk Mark results of the Adata on some hi end MSI board (I think it was), ANYWAY, I noted the "numbers", and did a Crystal test on mine, and wasn't really that close...all I can think is there is a performance hit, having to use NVMe's via a PCI-e adapter.

Having said that, it was good of byteshare to contribute to the data, and the Adata is only a little "slower" than his "basic" 970's.

I also tried several different PCI-e slots, to no avail, and I created a "Spanned" set in Disk Manager, and run your test, didn't make a **** of difference, either...wierd thing, even though they were suppose to be spanned, when I ran the test, it only seemed to be accessing one "drive", not both (as per blinking led's)

Anyway, you get what you pay for :( forget the hype !!!

[Atak_Snajpera]

Crystal disk and assd are also useless because by default they use 1GiB test file. That's still enough to fit in ssd's dram cache. Most reviewers on YouTube do not bother to change that option.

[guest]

Crystal disk and assd are also useless because by default they use 1GiB test file. That's still enough to fit in ssd's dram cache. Most reviewers on YouTube do not bother to change that option.

Fair enough, but I did notice that, and did a couple of 32Gb runs...I think it slowed it down a bit.

Anyway, I've had enough of testing...I did a couple of "test's" with RB.

I noticed (with a stop watch), that the NVMe's DO speed up the copying & indexing in the initial stage of job creation.

Then the whole point of this saga...the combining of the chunks & the muxing is a LOT faster.

So I will run a few of my "big ass" 4K encodes, and see how they handle them.

If I decide to throw some NVMe's into another PC, I will NOT get ADATA's again. Even 970's would be better, and the 970 Plus's would be a little again..Pro are a bit exy. :(

Change of subject..

I'm thinking of possibly utilising the new script you have added to RB.

Could you please give me some pointer's on how I could use that to move a finished encode off the NVMe, to another location.

eg:- from G:/Ripbot Encodes to D:/

[Atak_Snajpera]

I'm thinking of possibly utilising the new script you have added to RB.
Could you please give me some pointer's on how I could use that to move a finished encode off the NVMe, to another location.
So you find it useful after all... Who would have thought ;)

here is the script
move /Y "%OUTPUTFILEDIR%\%OUTPUTFILENAME%.%OUTPUTFILEEXT%" "D:\"

[guest]

So you find it useful after all... Who would have thought ;)

here is the script
move /Y "%OUTPUTFILEDIR%\%OUTPUTFILENAME%.%OUTPUTFILEEXT%" "D:\"

Yeah, I know....you've got a good memory. :rolleyes:

I guess what changed my mind was using it for something other than deleting job's, I'd still think I'd prefer to do that manually.

Thanks for that, I'll have a "play", but to ask a noob question, do I need to rename anything here ??

And is this added to the .ini file ??

Very little experience with "command's".

[Atak_Snajpera]

Thanks for that, I'll have a "play", but to ask a noob question, do I need to rename anything here ??
No. Those %VARIABLES% will be automatically replaced by ripbot264.

And is this added to the .ini file ??
You add this to Job finished successfully.cmd

[guest]

No. Those %VARIABLES% will be automatically replaced by ripbot264.


You add this to Job finished successfully.cmd

Yes, I had a look in there, after I sent the question.

Excellent, will give it a whirl.

Cheers & Thanks again.

BTW, I finished my crf 12, crf 16 & 30mb/s test's on a 1080 HEVC..I think the crf 12 has the edge, but they all look better than original. :)

[mparade]

Can you reach the share manually from the remote PC?
Can you ping the remote PC?

Admin could ping the remote PC and told everything seems to be ok with my VPN connection. I did not check the first one yet. Is there something more to set within from Ripbot for things to work?

Thank you. For 4K encoding I would definitely would like to utilizie the additional 16 cores.

[byteshare]

While not what we're doing but related to speed tests:
M.2 NVMe vs SSD vs OPTANE+HDD vs HDD Game Loading Times
https://youtu.be/2BS_6xOf0mw

Admin could ping the remote PC and told everything seems to be ok with my VPN connection. I did not check the first one yet. Is there something more to set within from Ripbot for things to work?

Thank you. For 4K encoding I would definitely would like to utilizie the additional 16 cores.
Check if you can manually reach the share since you can ping the computer. Might need to just change the permissions on the share and/or add the login info into RipBot's DE settings.

[GZZ]

having a problem with HDR10 on Red Heat. It seems to not honor the HDR10 encoding properly from the source.

I use the same settings as on other movies where it works just fine. But it seems like some info is missing in the reencoded. (marked with red line)

MediaInfo from Source:


Video
ID : 1
ID in the original source medium : 4113 (0x1011)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1 h 44 min
Bit rate : 63.0 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 24.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.316
Stream size : 45.8 GiB (94%)
Language : English
Default : No
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : BT.2020
Mastering display luminance : min: 0.0001 cd/m2, max: 1000 cd/m2
Original source medium : Blu-ray


MediaInfo from Reencode

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1 h 44 min
Bit rate : 19.5 Mb/s
Width : 3 840 pixels
Height : 2 076 pixels
Display aspect ratio : 1.85:1
Frame rate mode : Constant
Frame rate : 24.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.102
Stream size : 14.2 GiB (82%)
Writing library : x265 3.2+7-37648fca915b:[Windows][GCC 9.2.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=3 / numa-pools=+ / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2076 / interlace=0 / total-frames=2880 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=24 / keyint=240 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=40 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=1 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant


Reencode script:

@echo off
@echo ""\\DESKTOP_8700K\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\DESKTOP_8700K\RipBot264temp\job7\Chunks\1.avs" -strict -1 -f yuv4mpegpipe - | "\\DESKTOP_8700K\Ripbot264temp\tools\x265\x265_x64.exe" --seek 0 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --crf 18 --fps 24 --min-keyint 24 --keyint 240 --frames 2880 --sar 1:1 --profile main10 --output-depth 10 --ctu 64 --y4m --pools "+" --output "\\DESKTOP_8700K\RipBot264temp\job7\Chunks\1.265" -"
"\\DESKTOP_8700K\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\DESKTOP_8700K\RipBot264temp\job7\Chunks\1.avs" -strict -1 -f yuv4mpegpipe - | "\\DESKTOP_8700K\Ripbot264temp\tools\x265\x265_x64.exe" --seek 0 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --crf 18 --fps 24 --min-keyint 24 --keyint 240 --frames 2880 --sar 1:1 --profile main10 --output-depth 10 --ctu 64 --y4m --pools "+" --output "\\DESKTOP_8700K\RipBot264temp\job7\Chunks\1.265" -


Normally I see a line like this: "--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" but it seems to be missing.

Also from the x265 encoding log its missing a line.
It normally has this line, but not in this case: x265 [warning]: Turning on repeat-headers for HDR compatibility


y4m [info]: 3840x2076 fps 24000/1000 i420p10 sar 1:1 unknown frame count
raw [info]: output file: \\DESKTOP_8700K\RipBot264temp\job7\Chunks\10.265
x265 [info]: HEVC encoder version 3.2+7-37648fca915b
x265 [info]: build info [Windows][GCC 9.2.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main 10 profile, Level-5 (Main tier)
x265 [info]: Thread pool created using 12 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 3 / wpp(33 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 24 / 240 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing lslices=8 deblock sao
x265 [info]: frame I: 38, Avg QP:17.39 kb/s: 45065.87
x265 [info]: frame P: 805, Avg QP:18.67 kb/s: 25801.51
x265 [info]: frame B: 2037, Avg QP:21.68 kb/s: 8957.98
x265 [info]: Weighted P-Frames: Y:2.9% UV:2.5%
x265 [info]: consecutive B-frames: 10.7% 15.1% 12.1% 46.3% 15.9%


Sample: https://www.dropbox.com/s/sb81qwfeyu712th/Red%20Heat_Sample.mkv?dl=0

[Atak_Snajpera]

Sample: https://www.dropbox.com/s/sb81qwfeyu...ample.mkv?dl=0
Ok thanks...

BTW. I do not think that encoding such old movie in 3840x2076 makes any sense.
cropped frame 2160p
https://i.imgsafe.org/87/87244d2eb6.png

cropped frame 2160p -> downsampled to 1080p -> upsampled to 2160p
https://i.imgsafe.org/87/8724361bc6.png

It is another so called fake 4K. If I was you I would downsample to 1080p and activate MDegrain2. You will have much smaller file size and cleaner image.

[GZZ]

Ok thanks...

BTW. I do not think that encoding such old movie in 3840x2076 makes any sense.
cropped frame 2160p
https://i.imgsafe.org/87/87244d2eb6.png

cropped frame 2160p -> downsampled to 1080p -> upsampled to 2160p
https://i.imgsafe.org/87/8724361bc6.png

It is another so called fake 4K. If I was you I would downsample to 1080p and activate MDegrain2. You will have much smaller file size and cleaner image.

It was rescanned from the 35mm film to 4k in the 2019 remaster which the UHD is from: https://www.imdb.com/title/tt0095963/technical?ref_=tt_dt_spec
Digital Intermediate (4K) (2019 remaster)

But I was more into keeping the HDR flag (4k or 2k)

[Atak_Snajpera]

It was rescanned from the 35mm film to 4k in the 2019 remaster which the UHD is from: https://www.imdb.com/title/tt0095963/technical?ref_=tt_dt_spec
Digital Intermediate (4K) (2019 remaster)
My test clearly shows that you do not get true pixels. It looks like 2k upscale to 4k with added noise to fool you that It has more details.

[Ryushin]

Hi Atak,

Still encountering the bug with pulling in a MKV that has the TrueHD stream before the AC3 stream:
FFAudioSource: No audio track found
(D:\Temp\RipBot264temp\job19\getinfo.avs, line 4)

This is the default that MakeMKV creates as I need to use it to make a MKV file to pull into Ripbot264 if I have a seamless branching 4K. I'm having to remux it to put the AC3 track first before the TrueHD track and then it pulls in fine.

[Atak_Snajpera]

Hi Atak,

Still encountering the bug with pulling in a MKV that has the TrueHD stream before the AC3 stream:
FFAudioSource: No audio track found
(D:\Temp\RipBot264temp\job19\getinfo.avs, line 4)

This is the default that MakeMKV creates as I need to use it to make a MKV file to pull into Ripbot264 if I have a seamless branching 4K. I'm having to remux it to put the AC3 track first before the TrueHD track and then it pulls in fine.

Can you send me a sample for testing purposes?

[Ryushin]

Can you send me a sample for testing purposes?

I'll try and get you a sample. Not sure if I can chop a Atmos TrueHD stream or not.

I know electronics are crazy expensive where you are, just wondering if you've been able to move to 4K ripping and encoding for yourself personally yet.

[Ryushin]

I'll try and get you a sample. Not sure if I can chop a Atmos TrueHD stream or not.

I've tried trimming the mkv several times with ffmpeg and I can import the trimmed version, but not the original 4K version. Frustrating. Not sure what to do. I'll try a few more tests.

[GZZ]

I've tried trimming the mkv several times with ffmpeg and I can import the trimmed version, but not the original 4K version. Frustrating. Not sure what to do. I'll try a few more tests.

Dont know if I did it correctly. But I used MKVToolNix and set the output to be devided by size and choose 350mb output files. Then it divide the file into 350mb chunks, but they are not equal in size, so I think it does it by keyframe or something. Then I choose one of the file as a sample and delete the others.

[byteshare]

Dont know if I did it correctly. But I used MKVToolNix and set the output to be devided by size and choose 350mb output files. Then it divide the file into 350mb chunks, but they are not equal in size, so I think it does it by keyframe or something. Then I choose one of the file as a sample and delete the others.
Yes, MKVtoolnix will use key frames if you're cutting by size, duration, and even frames.
I've tried trimming the mkv several times with ffmpeg and I can import the trimmed version, but not the original 4K version. Frustrating. Not sure what to do. I'll try a few more tests.
Did you try MKVtoolnix?

[guest]

Would it be too much to ask if you could implement an option where you could send a shut down/kill command thru to an individual server's IP address, when ever you wanted or needed to ??

Preferably within the Encoding Client window/screen.

Very handy when the servers are elsewhere !!

Separate to the shut down options at the end of an encode.

I've had another idea about "power saving" during long "wait's" between encodes, in reference to DE, and several server pc's.

Now I have no idea if someone else has thought of this, and not too sure how it would work, anyways.

IF, you had the "remote" DE servers set to go to "sleep" once there were off a high cpu usage encode, would RB be able to "wake them" out of that sleep, once it started encoding the next job ??

[Ryushin]

Hi Atak,

Would is be possible to add a "Add Padding" tick box to the resize video. So selecting a setting like HDReady (720p) will scale the video and keep the aspect ratio of the video (keeps cropping settings) to fit within the resize video window. Then clicking the Padding box will add boarders if necessary to make the video fit the 720p range.

There also an issue with typing in the box to manually enter the size. I'm trying to type 960x720 (4x3 movies) and it keeps auto-correct the size to make it fit within a parameter before I can finish typing the size. The only way for me to do it is to copy 960 to the clipboard and paste it into the box.
or selecting a different audio source.or selecting a different audio source.

I wanted to wait to ask about an issue until I upgraded to SSDs on my server which I just completed yesterday. I bought six Crucial MX500 2TB drives and put then in a ZFS raidz2 (like raid6). I'm waiting on Supermicro to release a new Epyc Rome motherboard before upgrading the CPU (which is an old 4 core Xeon) to the 24 core Epyc. My RipBot server is a KVM Virtual Machine running on the linux server and it's only acting as a director to pull in the jobs and use DE to send them to other servers for encoding. It's virtual drive is now running on the SSD storage pool.

Working with HD sources is fairly quick and it only takes a few seconds when it's "Gathering Information" when leaving the Avisynth window. 4K sources on the other hand, takes minutes. Selecting a second audio source also takes a very long time.

Another issue is having it do auto cropping. I'll use this option for HD sources which might take a minute. 4K sources can be a 15-30 minute wait when having it do auto cropping. For 4K sources I'm selecting manual and entering the numbers manually.or selecting a different audio source.or selecting a different audio source. Even selecting New Frame takes a very long time to pull in.

[Atak_Snajpera]

Would is be possible to add a "Add Padding" tick box to the resize video. So selecting a setting like HDReady (720p) will scale the video and keep the aspect ratio of the video (keeps cropping settings) to fit within the resize video window. Then clicking the Padding box will add boarders if necessary to make the video fit the 720p range.
I will see...

There also an issue with typing in the box to manually enter the size. I'm trying to type 960x720 (4x3 movies) and it keeps auto-correct the size to make it fit within a parameter before I can finish typing the size. The only way for me to do it is to copy 960 to the clipboard and paste it into the box.
OK. I see the problem...

Working with HD sources is fairly quick and it only takes a few seconds when it's "Gathering Information" when leaving the Avisynth window. 4K sources on the other hand, takes minutes. Selecting a second audio source also takes a very long time.

Another issue is having it do auto cropping. I'll use this option for HD sources which might take a minute. 4K sources can be a 15-30 minute wait when having it do auto cropping. For 4K sources I'm selecting manual and entering the numbers manually.or selecting a different audio source.or selecting a different audio source. Even selecting New Frame takes a very long time to pull in.
On my real win7 and Xeon E5-2690 with 64GiB of RAM autocropping takes ~12s with UHD bluray source and ~3s with regular avc bluray source.

[Ryushin]

Dont know if I did it correctly. But I used MKVToolNix and set the output to be devided by size and choose 350mb output files. Then it divide the file into 350mb chunks, but they are not equal in size, so I think it does it by keyframe or something. Then I choose one of the file as a sample and delete the others.

I tried this and I'm afraid I still cannot duplicate the issue with a small trimmed file. Only the original it seems. I guess I'll just need to deal with this issue for the time being. The bigger issue though is the seamless branching in 4K sources causing Ripbot to cause audio sync problems. At least I have a workaround. Ripbot is still amazing software and I use it just about daily.

[Ryushin]

I will see...
On my real win7 and Xeon E5-2690 with 64GiB of RAM autocropping takes ~12s with UHD bluray source and ~3s with regular avc bluray source.

So I'm probably CPU bound then for the auto cropping. My seven year old Xeon E5-2609 at 2.40GHz with three of the four cores passed through to the VM is probably the issue with the cropping then. I'll do some more testing in a couple of months or so once Supermicro releases a new board and I get the 24 core Epyc Rome.

Do you think I'm still CPU bound for the Gathering Information slowness? Before it looked like I was I/O bound and looking at the Task Manager that is no longer the case. CPU is running around 75% during this time. But since there are only three cores, that could be an issue.

[Atak_Snajpera]

So I'm probably CPU bound then for the auto cropping. My seven year old Xeon E5-2609 at 2.40GHz with three of the four cores passed through to the VM is probably the issue with the cropping then. I'll do some more testing in a couple of months or so once Supermicro releases a new board and I get the 24 core Epyc Rome.

Do you think I'm still CPU bound for the Gathering Information slowness? Before it looked like I was I/O bound and looking at the Task Manager that is no longer the case. CPU is running around 75% during this time. But since there are only three cores, that could be an issue.

I would blame virtual machine instead of cores. 12s vs minutes looks weird even on 4C.

[byteshare]

IF, you had the "remote" DE servers set to go to "sleep" once there were off a high cpu usage encode, would RB be able to "wake them" out of that sleep, once it started encoding the next job ??
Are you still seeing huge waits between muxing at the end of jobs?
From my usage of RB, the time between jobs is not a lot. At most I could see a reason for another job to start once all of the chunks are done or in process, but even that wouldn't be a major difference IMO, unless you had a huge server farm, but at that point (no offense to RB) you should use different software or have more individual sessions of RB going.

[Ryushin]

I would blame virtual machine instead of cores. 12s vs minutes looks weird even on 4C.

Though what is stranger is the huge amount of difference it is between HD and 4K. Is there an option to use a different decoder for the auto detect boards and new frame option. I notice things got really slow for me around the time lsmash was implemented.

The VM has been fine, but a little slow in general, but overall, it got the job done. Now though the Gathering Information on anything related to 4K sources seems to take a very long time.

Oh, one other thing, is it still necessary to convert .sup to .sub when rendering subtitles?

[slalom]

Oh, one other thing, is it still necessary to convert .sup to .sub when rendering subtitles?
Uncheck the box in Main Settings

[GZZ]

I tried this and I'm afraid I still cannot duplicate the issue with a small trimmed file. Only the original it seems. I guess I'll just need to deal with this issue for the time being. The bigger issue though is the seamless branching in 4K sources causing Ripbot to cause audio sync problems. At least I have a workaround. Ripbot is still amazing software and I use it just about daily.

Did you extract it using MakeMKV ? I have noticed that on Animation like Disney and Pixar this issue occure. I found out there is a glitch in the movie when extracted with MakeMKV (I think its extraction issue in makeMKV)

To find the glitch
What I did to find it, is to run MKVToolNix on the MKV file and then try load all the chunks into Ripbot264 (Use batch mode), one the chunks will fail with that error (no audio), it did on mine. Then try play this chunk and see if there is a glitch in the audio (you can hear it).

To fix it
Use MkvToolNix on your bluray/UHD source to extract the movie to mkv and then try load it into ripbot264

[Ryushin]

Uncheck the box in Main Settings

That is unchecked and muxing in subtitles works correctly. Rendering subtitles always converts sup to sub regardless of the setting in Main Settings.

[Ryushin]

Did you extract it using MakeMKV ? I have noticed that on Animation like Disney and Pixar this issue occure. I found out there is a glitch in the movie when extracted with MakeMKV (I think its extraction issue in makeMKV)

To find the glitch
What I did to find it, is to run MKVToolNix on the MKV file and then try load all the chunks into Ripbot264 (Use batch mode), one the chunks will fail with that error (no audio), it did on mine. Then try play this chunk and see if there is a glitch in the audio (you can hear it).

To fix it
Use MkvToolNix on your bluray/UHD source to extract the movie to mkv and then try load it into ripbot264

I did not know MkvToolNix could make a mkv from a BD/4K source. I would prefer MkvToolNix to do the work.

[guest]

Are you still seeing huge waits between muxing at the end of jobs?
From my usage of RB, the time between jobs is not a lot. At most I could see a reason for another job to start once all of the chunks are done or in process, but even that wouldn't be a major difference IMO, unless you had a huge server farm, but at that point (no offense to RB) you should use different software or have more individual sessions of RB going.

TBH, I haven't done any 4K encodes since getting the NVMe's, but that's not really what info I was after.

I know there's an option for RB to "wake up" DE server's, but I haven't had any success with that...

So despite how long the "wait" times are between encodes, there is STILL a period where the servers aren't doing anything, and as in my case, I have 6 hungry servers eating up power, when they could be asleep, or even better, OFF.

Under full load the "farm" uses approx 2400W, and I guess when they're idling, it would be maybe a 1/4 of that, so lets say 600W for say 10 - 15 minutes, several times a day, adds up to a lot of wasted energy.

you should use different software or have more individual sessions of RB going.

So what different software can do what RB can do ??

And AFAIK, the only way to have individual sessions of RB, is to have another client pc "sharing" the "farm", but that kinda defeats the purpose of doing large encodes using the whole farm for a process as fast as possible.

[byteshare]

TBH, I haven't done any 4K encodes since getting the NVMe's, but that's not really what info I was after.

I know there's an option for RB to "wake up" DE server's, but I haven't had any success with that...

So despite how long the "wait" times are between encodes, there is STILL a period where the servers aren't doing anything, and as in my case, I have 6 hungry servers eating up power, when they could be asleep, or even better, OFF.

Under full load the "farm" uses approx 2400W, and I guess when they're idling, it would be maybe a 1/4 of that, so lets say 600W for say 10 - 15 minutes, several times a day, adds up to a lot of wasted energy.
Seems reasonable to me (the amount of power waste), but have you been able to suspend/sleep your computers and wake them remotely with other software? Might not be a RB issue. Not all of my computers fully support it.

So what different software can do what RB can do ??

And AFAIK, the only way to have individual sessions of RB, is to have another client pc "sharing" the "farm", but that kinda defeats the purpose of doing large encodes using the whole farm for a process as fast as possible.
The full quote of what I said is important IMO
unless you had a huge server farm, but at that point (no offense to RB) you should use different software or have more individual sessions of RB going.
Talking about a major server farm, which you don't have. There is other software but talking at a commercial level, so nothing free.
Are you trying to do it as fast as possible or trying to maximize CPU usage all of the time? I thought you wanted to maximize CPU usage, and in that case another RB sharing the server pool would be helpful.
If you want it as fast as possible than the way RB is now is better because you have every server waiting at the ready for the next job.
Like we've talked about the only thing that could speed that up a little is if another job started while the last job finished the muxing, but depending on your job and setup that can be a very minimal amount of time or like how you were doing it a lot of time.

[guest]

Seems reasonable to me (the amount of power waste), but have you been able to suspend/sleep your computers and wake them remotely with other software? Might not be a RB issue. Not all of my computers fully support it.

I have tried a couple of basic things, that seem to work.

Several years ago I installed a small app that monitored CPU usage (among other things), and you could set it up to either sleep, hibernate, or shutdown, once cpu usage dropped below a certain level, but I can't remember what it was :(

this:- http://www.ampsoft.net/utilities/WinOFF.php#Download

I haven't tried a "sleep or hibernate" function yet, I was just "putting it out there"

So how do you set up RB to "wake up" a server ??, I have the options enabled in RB, I have all the MAC address & username's etc, but I have no success (but I haven't tried that recently)

The full quote of what I said is important IMO

Sorry if I've upset you, but I did have the full quote, in the previous quote, and decided just to use that sentence for the next comment..check.

Talking about a major server farm, which you don't have. There is other software but talking at a commercial level, so nothing free.
Are you trying to do it as fast as possible or trying to maximize CPU usage all of the time? I thought you wanted to maximize CPU usage, and in that case another RB sharing the server pool would be helpful.
If you want it as fast as possible than the way RB is now is better because you have every server waiting at the ready for the next job.
Like we've talked about the only thing that could speed that up a little is if another job started while the last job finished the muxing, but depending on your job and setup that can be a very minimal amount of time or like how you were doing it a lot of time.

Yes, I want to maximise CPU usage, to do it faster....

Not sure why you're suggesting another client doing other jobs....(which I do anyway), BUT on "big ass" encodes, 1 client, ALL the DE servers dedicated to that one job...isn't that utilising CPU usage, and getting it done as fast as possible ??? or am I missing something here.

[mparade]

1) run encodingserver.exe on remote pc and take a screenshot.
2) run ripbot264.exe on your home pc. go to settings->Distributed encoding and show me what client ips are listed under CLIENT IP
3) Are you sure firewall on remote pc does block connection?

This is the screenshot from home pc:

http://mammutmail.com/hu/download/b8303cc66fb21282c2be3d994bd7ec1e

[Atak_Snajpera]

This is the screenshot from home pc:

http://mammutmail.com/hu/download/b8303cc66fb21282c2be3d994bd7ec1e

Have you checked firewall settings on your Work-PC? If EncodingClient can't connect to server (assuming that IP reported by EncodingServer is the same as you typed in RipBot264 settings) then I would suspect firewall issue.

[mparade]

This is the screenshot from work pc:

http://mammutmail.com/hu/download/c668db124bc39be7e763ac5cd6bacaf4

I have double checked that firewall on work-PC has been switched off.

Server reported some "socket" problem right after it made contact with EncodingClient.

[Atak_Snajpera]

Server reported some "socket" problem right after it made contact with EncodingClient.
Too bad you didn't take a screenshot... That message is important.

[GZZ]

My test clearly shows that you do not get true pixels. It looks like 2k upscale to 4k with added noise to fool you that It has more details.

Hi Atak_Snajpera
Did you find a solution for the HDR issue ?

[Atak_Snajpera]

Hi Atak_Snajpera
Did you find a solution for the HDR issue ?

yes...

[mparade]

Too bad you didn't take a screenshot... That message is important.

There you are :):

http://mammutmail.com/hu/download/f12357709850442d2867fce26e0afa38

[Atak_Snajpera]

There you are :):

http://mammutmail.com/hu/download/f12357709850442d2867fce26e0afa38

Your home pc blocks connection from encodingserver. ( Connection timeout). Connection must be established in both ways. Check firewall settings on home pc.

[mparade]

After switching off firewall completely on home-pc result is still the same. Should I reopen encodingserver.exe and reconfigure servers on work-pc?

Now I got this error message on work-pc:

http://mammutmail.com/hu/download/2f69b2770124bed00cf2ab46cbf28579

[duffbeer]

Though what is stranger is the huge amount of difference it is between HD and 4K. Is there an option to use a different decoder for the auto detect boards and new frame option. I notice things got really slow for me around the time lsmash was implemented.

The VM has been fine, but a little slow in general, but overall, it got the job done. Now though the Gathering Information on anything related to 4K sources seems to take a very long time.

Oh, one other thing, is it still necessary to convert .sup to .sub when rendering subtitles?

You're not alone. I reported this issue quite a while ago and nothing has changed.
I have a 9900K based PC that is a dedicated encoder. I have the same delay issues as you. If I go back to 1.24.1 everything is OK.
Right now it takes me about 90 mins to queue up a 4K source. Cropping seems to be really slow.

[Atak_Snajpera]

After switching off firewall completely on home-pc result is still the same. Should I reopen encodingserver.exe and reconfigure servers on work-pc?

Now I got this error message on work-pc:

http://mammutmail.com/hu/download/2f69b2770124bed00cf2ab46cbf28579

You are still behind some extra firewall/antivirus on home-pc. There is some software on your PC or settings which actively refuse connection. In first case (socket error 10060) connection was totally ignored hence timedout message. Now after disabling your firewall you get at least different error message. I suspect that extra "layer of protection" (different software) is now rejecting incoming connection with proper message.

[Ryushin]

You're not alone. I reported this issue quite a while ago and nothing has changed.
I have a 9900K based PC that is a dedicated encoder. I have the same delay issues as you. If I go back to 1.24.1 everything is OK.
Right now it takes me about 90 mins to queue up a 4K source. Cropping seems to be really slow.

90 minutes seems about same for me as well and about 15 minute chunks as well.
Pull in a 4K: Long time, but this seem CPU bound now, so no complaints here.
Second Audio THD source (Gathering Information): 15 Minutes
Autocrop Frames: 15-30 minutes
Leave Avisysnth Windows (Gathering Information) 5-15 minutes

Waiting to buy a Epyc rome processor. Then I'll do tests again. I'm currently tuning my VMs for the new SSD raidz2. Don't expect to see much improvement there as the bottle neck seems to be CPU bound now.

[duffbeer]

90 minutes seems about same for me as well and about 15 minute chunks as well.
Pull in a 4K: Long time, but this seem CPU bound now, so no complaints here.
Second Audio THD source (Gathering Information): 15 Minutes
Autocrop Frames: 15-30 minutes
Leave Avisysnth Windows (Gathering Information) 5-15 minutes

Waiting to buy a Epyc rome processor. Then I'll do tests again. I'm currently tuning my VMs for the new SSD raidz2. Don't expect to see much improvement there as the bottle neck seems to be CPU bound now.

I don't think it is CPU bound. I don't run any VMs on my PC - it is a 9900K and has nothing else installed on it.
I would like to know how some people are using 1.25 without setup speed issues and some are not.
1.24.1 was fine so the latest version has been broken as far as I'm concerned.

[byteshare]

I don't think it is CPU bound. I don't run any VMs on my PC - it is a 9900K and has nothing else installed on it.
I would like to know how some people are using 1.25 without setup speed issues and some are not.
1.24.1 was fine so the latest version has been broken as far as I'm concerned.
On a Ryzen 1700 and SSD it doesn't take that much time. Are you hitting 100% CPU when the auto crop is going?


So how do you set up RB to "wake up" a server ??, I have the options enabled in RB, I have all the MAC address & username's etc, but I have no success (but I haven't tried that recently)
If you fill in the MAC address info it should be working. If it is not, test that you can get your PC to wake up remotely without RB but either another app or application.


Sorry if I've upset you, but I did have the full quote, in the previous quote, and decided just to use that sentence for the next comment..check.
Didn’t upset me but without the rest of the context it seemed like a completely different point.


Yes, I want to maximise CPU usage, to do it faster....
Not sure why you're suggesting another client doing other jobs....(which I do anyway), BUT on "big ass" encodes, 1 client, ALL the DE servers dedicated to that one job...isn't that utilising CPU usage, and getting it done as fast as possible ??? or am I missing something here.
I’m saying if you want it the fastest it is already setup to go pretty fast, once you solved the muxing problem. I’m saying if you want it to be more efficient then setting up a second RB would solve that problem. I was saying if you want both since the Dev doesn’t have an interest right now in changing how the jobs work, than you’d likely need some expensive software to do that. I’m not sure exactly your need for both fast and efficient but I am happy with just efficiency in RB, which is why I use it over other software.

[mparade]

You are still behind some extra firewall/antivirus on home-pc. There is some software on your PC or settings which actively refuse connection. In first case (socket error 10060) connection was totally ignored hence timedout message. Now after disabling your firewall you get at least different error message. I suspect that extra "layer of protection" (different software) is now rejecting incoming connection with proper message.

Ok, thanks. I don't have any idea. Firewall switched off completely on both machines + neither of them has antivirus installed.
This is what I still get:

[2019-11-04 20:49:22] Client established connection with server.
[2019-11-04 20:49:22] Command "CONNECT" received from client.
[2019-11-04 20:49:22] Connecting with client... (192.168.88.178:1011)
[2019-11-04 20:49:24] Socket Error # 10061
Connection refused.

[Atak_Snajpera]

By the way. Are you using VPN built in windows or you use other custom software? What are other ips detected in client IP combobox? What does Adapter description show for each extra IP?

[guest]

If you fill in the MAC address info it should be working. If it is not, test that you can get your PC to wake up remotely without RB but either another app or application.

I had a bit of a play around with this yesterday, and it seems that some of my SMC boards don't support the S3 power settings in BIOS, so it won't wake, but I can at least shut them down remotely.


Didn’t upset me but without the rest of the context it seemed like a completely different point.

OK, good, and fair enough, I guess I thought I didn't need to double up on the quotes..


I’m saying if you want it the fastest it is already setup to go pretty fast, once you solved the muxing problem. I’m saying if you want it to be more efficient then setting up a second RB would solve that problem. I was saying if you want both since the Dev doesn’t have an interest right now in changing how the jobs work, than you’d likely need some expensive software to do that. I’m not sure exactly your need for both fast and efficient but I am happy with just efficiency in RB, which is why I use it over other software.

I guess I'll work with what I've got, I am yet to do a "big ass" 4K encode, but I think the NVMe's will make a big difference at the end of the process, not so much at the start.

I'm puzzled by the long times that other user's are talking about, I might do some test's and post the results.

Cheers

[Ryushin]

Atak,

Going to buy a temporary CPU from Ebay. Wondering if it's more important to have hertz or cores for Ripbot. Looking at a 6 core 2.6Ghz or a 8 core 2.0Ghz.

Found out my current CPU goes for $6.00 now on Ebay. LOL

[Atak_Snajpera]

Atak,

Going to buy a temporary CPU from Ebay. Wondering if it's more important to have hertz or cores for Ripbot. Looking at a 6 core 2.6Ghz or a 8 core 2.0Ghz.

Found out my current CPU goes for $6.00 now on Ebay. LOL

2.6 * 6 = ~16
2.0 * 8 = 16
In this case i would choose higher clock .

[mparade]

By the way. Are you using VPN built in windows or you use other custom software? What are other ips detected in client IP combobox? What does Adapter description show for each extra IP?

Admin installed ZyWALL IPSec VPN Client on my home-pc.
10.10.10.10 the other ip detected in client IP combobox.
Adapter description includes two options regardless of ip chosen in client IP combobox:

1.Intel(R) I211 Gigabit Network Connection
2. TheGreenBow Virtual Miniport Adapter

[Ryushin]

Admin installed ZyWALL IPSec VPN Client on my home-pc.
10.10.10.10 the other ip detected in client IP combobox.
Adapter description includes two options regardless of ip chosen in client IP combobox:

1.Intel(R) I211 Gigabit Network Connection
2. TheGreenBow Virtual Miniport Adapter

Just to let you know, I've seen some VPN's tunnel ALL traffic down the VPN. Local traffic (such as network printers) could no longer be accessed. I don't like these types of VPNs at all and I set up firewalls and VPNs for a living. I understand their usage for extreme security, but not having access to your local network, c'mon.

[guest]

So I have noticed a few posts about very long times to start a 4K job, so I thought I'd do a couple of tests for myself.

I chose Harry Potter #1, which is a remux 4K, 60.7Mb/s, 84.5Gb mkv.

I also ran it thru MKVToolnix to rip out EVERYTHING but the video, which took 4:30 mins (Raid-0 to nvme)

Also wanted to see what difference there was between LSmash & FFMS2.

So, chose the untouched mkv to start with, and it took 4:30 mins to demux, and a further 4:20 mins until the job creation was ready, with LSmash.

Using FFMS2 it took 4:30 mins to demux, and further 3:50 mins 'til job was ready.

Then the stripped mkv took 0 mins to demux, and a total of 3:25 mins with LSmash.

Same file using FFMS2 took 0 mins to demux, and a total of 3:12 mins.

Once they were queued up, it was time to see how long it took to copy the file, get the chunks created, etc, etc.

The untouched job took 12:00 mins 'til the chunks were all ready to go.

With the stripped mkv, it took 8:45 mins...not forgetting the it took 4:30 mins to strip out the file in the first place, so it's close'ish.

All these figures are having the original file on a RAID-0 of 2 x Seagate 3Tb 7,200 rpm spinners, copying to an ADATA NVM'e, using a single Xeon 12c cpu.

Now, I cannot provide any final muxing, etc info at this time, as that will take a couple of days for me to get around to do that, but I WILL post my results.

So regardless of how long that's going to take, there WILL be a reasonable amount of time that all DE servers aren't doing anything.

[howzz]

@Atak_Snajpera

i noticed that in the latest update, you included an option for user to select decoding method. although Lmash is slow, on my NvMe it's still fast for demuxing, indexing etc.
but my question is, is there any real speed difference in using GPU for "decoding" method?

btw, i see a lot of folks are discussing about NvMe here. i was an early adapter of NvMe for my 4K workflow, trust me, invest in a couple good NvMe drives. it's worth it. and do yourself a favor, don't go below 1TB. my current 4K workflow, nothing takes longer than 4 mins for initial demux, indexing, gathering information altogether. most 4K UHD sources take 3~4 mins total to do all that. if you have the source sits on the same NvMe drive, it'll only take 3~4 mins.

the trick with shopping for NvMe drives is look for TLC drives and avoid QLC drives at all cost. QLC's raw writing speed is less than 100 MB/s. i've tried them all and went through them all and experience all of it the hard way. once the SLC cache run out, QLC's raw writing speed will fall below 100 MB/s sometimes 70 MB/s, that's slower than most high end 7200rpm spindle. and smaller the drive, smaller the SLC casche will be. on a 1TB QLC drive, your SLC cache is probably gonna be about less than 20GB when it's filled at 50%. the intel 660P 2TB drive, theoretically, if you over provision it and treat it as a 1.5TB drive, theoretically you'll always have more than 50GB of SLC caching. but just do yourself a favor, avoid all the QLC drives out there. go for a fast TLC drive from Corsair, Sillicon power, Samsung, and you'll be a lot happier. most TLC's raw writing speed, provided if the onboard processor is of recent version, will be around 600~800 MB/s even after it runs out of its SLC caching. so when doing 4K UHD workflow working with large 60~70 GB files, you're going to be counting on the raw writing speed. when shopping around, look for reviews that do tests on large 25~50 GB file "write" tests, and the chart will show you the speed difference once those SLC cache runs out.

if money is of no object, Samsung is the way to go. but otherwise get one of those Corsair MP510, or silicon power TLC drives. 1~2TB will be most ideal. and make sure that your mobo supports PCIe 3.0 x4 M.2, even though realistically, ripbot workflow will never really exceed 1500 MB/s throuput, just under the max bandwidth of PCIe 2.0 x4

[duffbeer]

So I have noticed a few posts about very long times to start a 4K job, so I thought I'd do a couple of tests for myself.

I chose Harry Potter #1, which is a remux 4K, 60.7Mb/s, 84.5Gb mkv.

I also ran it thru MKVToolnix to rip out EVERYTHING but the video, which took 4:30 mins (Raid-0 to nvme)

Also wanted to see what difference there was between LSmash & FFMS2.

So, chose the untouched mkv to start with, and it took 4:30 mins to demux, and a further 4:20 mins until the job creation was ready, with LSmash.

Using FFMS2 it took 4:30 mins to demux, and further 3:50 mins 'til job was ready.

Then the stripped mkv took 0 mins to demux, and a total of 3:25 mins with LSmash.

Same file using FFMS2 took 0 mins to demux, and a total of 3:12 mins.

Once they were queued up, it was time to see how long it took to copy the file, get the chunks created, etc, etc.

The untouched job took 12:00 mins 'til the chunks were all ready to go.

With the stripped mkv, it took 8:45 mins...not forgetting the it took 4:30 mins to strip out the file in the first place, so it's close'ish.

All these figures are having the original file on a RAID-0 of 2 x Seagate 3Tb 7,200 rpm spinners, copying to an ADATA NVM'e, using a single Xeon 12c cpu.

Now, I cannot provide any final muxing, etc info at this time, as that will take a couple of days for me to get around to do that, but I WILL post my results.

So regardless of how long that's going to take, there WILL be a reasonable amount of time that all DE servers aren't doing anything.

Now try it with full 4K UHD disc as the source. You will see how much longer it takes.

[guest]

Now try it with full 4K UHD disc as the source. You will see how much longer it takes.

Haven't got access to any "full" disk's, unless I acquire a UHD rip as an .iso.

But why would there be a big difference ?? Is it the ripping process that takes so long ??

Do you rip it first, or let RB do that (if it can) ??

Ripping Bluray disk's was a PITA, I'm guessin' 4K are SO much worse.

[Atak_Snajpera]

but my question is, is there any real speed difference in using GPU for "decoding" method?

IT is complicated..
https://i.imgsafe.org/28/286ce8b657.png

https://i.imgsafe.org/28/286ce86c97.png

https://i.imgsafe.org/37/37bfdbdafa.png

most TLC's raw writing speed, provided if the onboard processor is of recent version, will be around 600~800 MB/s even after it runs out of its SLC caching.
TLC is still pathetic in comparison to old good MLC. Write speed on Samsung 970 PRO is ~2.5 GiB/s (No SLC cache BS)!

[Atak_Snajpera]

Just to let you know, I've seen some VPN's tunnel ALL traffic down the VPN. Local traffic (such as network printers) could no longer be accessed. I don't like these types of VPNs at all and I set up firewalls and VPNs for a living. I understand their usage for extreme security, but not having access to your local network, c'mon.

Since you have been doing this VPN stuff for living I would like you to ask something.

When you connect to remote LAN via VPN then shouldn't home-pc get the same three sections of IP?

https://i.postimg.cc/Hsc4zwhm/screenshot-home-pc.png

https://i.postimg.cc/PfFQ13KK/socket-error-message.png

My common sense tells me that after connecting to remote LAN Client IP should be starting from 192.168.2.xxx

[byteshare]

Just to let you know, I've seen some VPN's tunnel ALL traffic down the VPN. Local traffic (such as network printers) could no longer be accessed. I don't like these types of VPNs at all and I set up firewalls and VPNs for a living. I understand their usage for extreme security, but not having access to your local network, c'mon.
Which VPN solution would you recommend for setting up RipBot to have remote servers?

[Atak_Snajpera]

Which VPN solution would you recommend for setting up RipBot to have remote servers?

Just use builtin in windows.

[NasaRacer]

Not sure if this is a known issue but on a few of my recent 4k UHD Mkvs, the file Demuxing and indexing are normal. Then during "gathering information" it fails with the message "FFAudioSource: No audio track found
(C:\Temp\RipBot264temp\job1\getinfo.avs, line 4)" So using MKVToolNix I made a shorter version of the same file 1min of the file and then another with 10min of the file. The 1min version works but the 10min version fails with the same error. Then I made a full version of the file with the video only and no audio track and that is working okay as well. I used XMedia Recode to encode just the audio and that worked correctly. So I am not sure what the issue is.
The audio is Dolby TrueHD 7.1 both movies in question are from Disney.

[Atak_Snajpera]

Not sure if this is a known issue but on a few of my recent 4k UHD Mkvs, the file Demuxing and indexing are normal. Then during "gathering information" it fails with the message "FFAudioSource: No audio track found
(C:\Temp\RipBot264temp\job1\getinfo.avs, line 4)" So using MKVToolNix I made a shorter version of the same file 1min of the file and then another with 10min of the file. The 1min version works but the 10min version fails with the same error. Then I made a full version of the file with the video only and no audio track and that is working okay as well. I used XMedia Recode to encode just the audio and that worked correctly. So I am not sure what the issue is.
The audio is Dolby TrueHD 7.1 both movies in question are from Disney.
Can you send me that not working 10 minute version?

[Ryushin]

Since you have been doing this VPN stuff for living I would like you to ask something.

When you connect to remote LAN via VPN then shouldn't home-pc get the same three sections of IP?

My common sense tells me that after connecting to remote LAN Client IP should be starting from 192.168.2.xxx

A lot of VPN software/gateways will use a IP subnet to connect to, then the VPN gateway will route the traffic to the appropriate network.

For example, I use OpenVPN for and Fortigate for my VPNs. The IP block that you use for the VPN users should be almost a random block because you do not want your VPN users to have the same block as you are running on your networks. In fact, to go further, any networks that are behind your VPNs, you will not want them to be commonly used for home networks.

I try and stay away from the 192.168.*.* blocks for business use. Most likely the home users are using 192.168.0.* and 192.168.1.*, but not always.

An example set up simple corporate network.
Internal Networks: 10.20.10.0/24
DMZ: 10.20.30.0/24
Secured Development: 10.20.40.0/24
VPN Network: 172.27.99.0/24

A road warrior or home user would connect to the VPN. They will get a 172.27.99.* for their VPN address. If I have configured a split VPN, then only the networks behind the VPN gateway will have routes added to the VPN client. The VPN client will still be able to access their local traffic and get access to the Internet without going through the VPN. If I configured Tunnel Mode VPN, and can force ALL traffic to go through the VPN or just allow the client to have access to their local network, but any traffic outside of their local network gets tunneled down the VPN.

There are some VPNs that will bridge traffic and make the VPN client seem like their are on one of the local work networks, but these are not done much any more.

Does RipBot assume everything is flat network (Layer2) or does it allow routing to take place (Layer 3) and jump networks to appropriate routed destination.

Does RipBot bind to single IP address or does it bind to all the local addresses on the machine. I wondering if that could be causing issues if it is binding to a single address as this will cause issues with VPNs.

Did I understand the question right?

As for what VPN I recommend, that would be OpenVPN. I would look at using a Raspberry Pi and a firewall distro, or a wireless router that comes bundled with a OpenVPN gui. Learning OpenVPN, especially, with everything it can do, is not for the faint of heart.

[Atak_Snajpera]

Does RipBot bind to single IP address or does it bind to all the local addresses on the machine. I wondering if that could be causing issues if it is binding to a single address as this will cause issues with VPNs.
It binds to one specified IP for encodingclient (selected in Client IP combobox) and encodingserver (selected via command line).

According to mparade post here https://forum.doom9.org/showthread.php?p=1889585#post1889585
other ip is 10.10.10.10 - TheGreenBow Virtual Miniport Adapter. Does not look like anything useful...

[howzz]

IT is complicated..
https://i.imgsafe.org/28/286ce8b657.png

https://i.imgsafe.org/28/286ce86c97.png

https://i.imgsafe.org/37/37bfdbdafa.png


TLC is still pathetic in comparison to old good MLC. Write speed on Samsung 970 PRO is ~2.5 GiB/s (No SLC cache BS)!

are these numbers in FPS i assume?

my suggestion for the TLC was because realistically, for cost reasons. But if money is no object, the Samsung Pro really is the way to go. but for those building large capacity 1TB/2TB drives for NvMe, TLC is a pretty fast compromise imo.

edit: are there even any 2TB samsung MLC NvMes? at least i can't find any.

[howzz]

and secondly, i forgot to ask. is there any quality lost if choosing GPU as the decoder as oppose to Lmash. :p
i mean decoding is decoding right? quality is affected during the x265 "encoding" calculations.

[Atak_Snajpera]

and secondly, i forgot to ask. is there any quality lost if choosing GPU as the decoder as oppose to Lmash. :p
i mean decoding is decoding right? quality is affected during the x265 "encoding" calculations.

No quality penalty. GPU just provides frames to encoder.

[Ryushin]

It binds to one specified IP for encodingclient (selected in Client IP combobox) and encodingserver (selected via command line).

According to mparade post here https://forum.doom9.org/showthread.php?p=1889585#post1889585
other ip is 10.10.10.10 - TheGreenBow Virtual Miniport Adapter. Does not look like anything useful...

Can you change Ripbot so there is an option to bind to all IPs on the local machine which I think should be the default? I also run IPv6, so it will be nice to just put a DNS name in instead of a IP. Also, me being me, to be able to bind to certain IPs is also very beneficial and I do that all the time for machines that are connected on multiple networks, especially if one of them is Internet facing and I want to only provide a service to internal clients.

[guest]

So today I was able to test a reasonably sized 4K movie..

I chose to do "Frozen", which I stripped out the audio, as I don't need/want TrueHD (turned out the thd track was corrupted anyway).

So the file ended up being 42.5Gb @ 56.8Mb/s, NO audio, or subs !!

After the encoding if was 35.25Gb @ 47.1Mb/s, using CRF 12.

After the initial loading of the job, it took a further 3 hrs:37 m to complete the encode.

The combining took 1:45 mins, and the muxing took 2:55 mins for a total of only 4:40 mins.

A lot better than this :-

..the combining of the chunks took 10 minutes 30, then the muxing process took a further 15 minutes 30 .. (from post #17533)

which was admittedly a larger file with all it's tracks, but on slow SSD's.

Now I would think that "Frozen" would have taken a little longer if it had to do EVERYTHING, but I did the final "mux" with MKVToolnix, to add audio & subs.

[guest]

So I'm doing some more 4K encoding today, and I have been having some problems with the connection to some of the DE servers, in the other room....wasn't happening yesterday....

I got these errors, and wondering what might be causing them :-

[7/11/2019 13:35:33] TCPClient7 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:35:47] TCPClient10 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:40:00] TCPClient8 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:40:00] TCPClient7 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:46:06] TCPClient12 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:46:30] TCPClient12 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:56:27] TCPClient13 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:57:53] TCPClient9 raised exception while attempting to sent command.(Not Connected)

But it seems to be going OK now, I think it takes a while for the process to "warm up".

Another thing I've noticed that might be associated with this, is some chunks will get to a certain %, and just stall there :(

And with all the chunks "flashing" is some way of showing that they are processing, it's a bit hard to focus in on which ones are doing this.

Not sure if the chunk "line's" are suppose to flash, but it would be a lot easier on the eyes if maybe they changed font size and/or colour, and NOT flash....just as well I'm not "epileptic".

And I have a feature request for chunks....it would be handy to have a small display of how many "chunks" are being processed, instead of having to count them.

And to top it off, how about some more DE servers, say another 4, at least. Need all the help I can get, and I have more pc's I could add to the 4K process.

And no I don't like the idea of running separate sessions with different pc's, that really doesn't speed up my process...I just want to get it done one "system", as fast as possible !!!

I do smaller basic encodes on separate pc's.

[mparade]

Just use builtin in windows.

Could you suggest something in my case for the VPN connection to work? Admin would like to help if problem is on "his side".

Thank you for the help.

[GZZ]

So I'm doing some more 4K encoding today, and I have been having some problems with the connection to some of the DE servers, in the other room....wasn't happening yesterday....

I got these errors, and wondering what might be causing them :-

[7/11/2019 13:35:33] TCPClient7 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:35:47] TCPClient10 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:40:00] TCPClient8 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:40:00] TCPClient7 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:46:06] TCPClient12 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:46:30] TCPClient12 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:56:27] TCPClient13 raised exception while attempting to sent command.(Not Connected)
[7/11/2019 13:57:53] TCPClient9 raised exception while attempting to sent command.(Not Connected)

But it seems to be going OK now, I think it takes a while for the process to "warm up".

Another thing I've noticed that might be associated with this, is some chunks will get to a certain %, and just stall there :(

And with all the chunks "flashing" is some way of showing that they are processing, it's a bit hard to focus in on which ones are doing this.

Not sure if the chunk "line's" are suppose to flash, but it would be a lot easier on the eyes if maybe they changed font size and/or colour, and NOT flash....just as well I'm not "epileptic".

And I have a feature request for chunks....it would be handy to have a small display of how many "chunks" are being processed, instead of having to count them.

And to top it off, how about some more DE servers, say another 4, at least. Need all the help I can get, and I have more pc's I could add to the 4K process.

And no I don't like the idea of running separate sessions with different pc's, that really doesn't speed up my process...I just want to get it done one "system", as fast as possible !!!

I do smaller basic encodes on separate pc's.

You should definitly look in your Windows system log and see if your netcard is disconnecting/reconnecting. I had a issue like yours when I had a bad driver for my 10Gbe netcard, upgrading to a newer driver helped. I think its more of a network issue then a problem in the application.

[guest]

You should definitely look in your Windows system log and see if your netcard is disconnecting/reconnecting. I had a issue like yours when I had a bad driver for my 10Gbe netcard, upgrading to a newer driver helped. I think its more of a network issue then a problem in the application.

I am only using the onboard 1 Gbe NIC's for DE, I doubt that there is any advantage of using a 10 Gbe, however, I am only using the Windows NIC drivers, so I will endeavour to update ALL the Intel drivers, so thanks for the suggestion.

I have to agree that it's certainly a network / cable problem, I had to swap out a patch cable yesterday, as it was dropping in & out, causing problems.

Just out of interest, what 10 Gbe NIC's are you running ?? do you use RJ45 or SFP+ connections ??

I'm gradually setting up a 10 Gbe LAN, and mainly using Mellanox NIC's. (but not for RB)

[Ryushin]

Could you suggest something in my case for the VPN connection to work? Admin would like to help if problem is on "his side".

Thank you for the help.

I think the first thing is to verify that you can connect to each process. From your Ripbot machine, install nmap (they have a windows version).

Then using nmap, see if the Encoding Server ports are open on your other machines. I would run command like this from Linux:
nmap -p 1000,2000,3000,4000 192.168.9.7

My results look like this:
nmap -p 1000,2000,3000,4000 192.168.9.7
Starting Nmap 7.80 ( https://nmap.org ) at 2019-11-07 05:22 MST
Nmap scan report for taffy (192.168.9.7)
Host is up (0.0023s latency).

PORT STATE SERVICE
1000/tcp open cadlock
2000/tcp filtered cisco-sccp
3000/tcp filtered ppp
4000/tcp filtered remoteanything

Nmap done: 1 IP address (1 host up) scanned in 1.25 seconds

I'm running one Encoding Server on that client. The scan shows port 1000 as open, and the other ports as filtered. Filtered will be either there is nothing listening on those ports or a firewall has filtered them.

That will at least tell you if you have some kind of network problem.

[byteshare]

Another thing I've noticed that might be associated with this, is some chunks will get to a certain %, and just stall there :(

And with all the chunks "flashing" is some way of showing that they are processing, it's a bit hard to focus in on which ones are doing this.

Not sure if the chunk "line's" are suppose to flash, but it would be a lot easier on the eyes if maybe they changed font size and/or colour, and NOT flash....just as well I'm not "epileptic".
Add "/restart-if-no-progress" to your startup script for the servers. Problem is that FFMPEG and/or AVISynth can get hung up.

[guest]

Add "/restart-if-no-progress" to your startup script for the servers. Problem is that FFMPEG and/or AVISynth can get hung up.

Hi byteshare,

I DO have that on EVERY PC, and tbh, I don't think it works that well.

[Atak_Snajpera]

Hi byteshare,

I DO have that on EVERY PC, and tbh, I don't think it works that well.

Do ffmpeg.exe and x26x.exe still use cpu when this happens?

[GZZ]

I am only using the onboard 1 Gbe NIC's for DE, I doubt that there is any advantage of using a 10 Gbe, however, I am only using the Windows NIC drivers, so I will endeavour to update ALL the Intel drivers, so thanks for the suggestion.

I have to agree that it's certainly a network / cable problem, I had to swap out a patch cable yesterday, as it was dropping in & out, causing problems.

Just out of interest, what 10 Gbe NIC's are you running ?? do you use RJ45 or SFP+ connections ??

I'm gradually setting up a 10 Gbe LAN, and mainly using Mellanox NIC's. (but not for RB)

First off DE encoding wont benefit from 10gbe at all and you properly need better cable (cat6a). I use it to gain better performance from my synology NAS. On avg it gives me 300-400mb/s and up to 700mb/s. So really nice when moving large uhd/bluray files.

I use RJ45 cable and a 8 port buffalo switch (mp2008).

[guest]

First off DE encoding wont benefit from 10gbe at all

I use RJ45 cable and a 8 port buffalo switch (mp2008).

Yes, I think I had mentioned already that 10 Gbe would be no advantage to RB....

I'm using SFP+ cables & NIC's, with a MikroTik 8 port switch.

You didn't mention what NIC's you're using...

[guest]

Do ffmpeg.exe and x26x.exe still use cpu when this happens?

TBH, that didn't even enter my mind, to check !!

I wouldn't really know what I was looking for, anyway.

I just watched the stalled chunk/server, for a couple of minutes, and when it hadn't progressed, I simply stopped it, and let another server take over.

I DID (once) have pop error msg stating that Encoder Stalled, and I think it was in relation to FFMPEG or similar, didn't stay up for long.

I'm using the very latest RB build, and I am going to update all my NIC drivers today, probably won't affect this...but good to do.

[GZZ]

Yes, I think I had mentioned already that 10 Gbe would be no advantage to RB....

I'm using SFP+ cables & NIC's, with a MikroTik 8 port switch.

You didn't mention what NIC's you're using...

Asus XG-C100C in my desktop and Intel x540-T2 in my nas.

[guest]

Asus XG-C100C in my desktop and Intel x540-T2 in my nas.

So just a simple P2P connection between them, then.

Are you planning to expand to other pc's ??

To get the best out of 10 Gbe, you need fast drives @ both ends.

I'm just starting out with 10 Gbe, and the transfer speeds vary a LOT.

[byteshare]

Update today, MKVTooNix update to 39.0.0, core 2019.11.08, and scripts 2019.11.08.
@Atak Anything that could affect us or more along the lines of bug fixes?

[GZZ]

So just a simple P2P connection between them, then.

Are you planning to expand to other pc's ??

To get the best out of 10 Gbe, you need fast drives @ both ends.

I'm just starting out with 10 Gbe, and the transfer speeds vary a LOT.

10GBE is more the for the fun of it. But I got 1GBE Fiber internet connection and if I use it close to 100%, then copying a file over the network will make the download/upload suffer. No problem with that on 10GBE. For the moment I dont plan on other expansions of my network.

Got 2 SSD in Raid 0 for Encoding and processing files. It gives me 1200mb/s of read/write speed. On my nas its just 4x10GB Ironwolf disc in Raid 5 (can be expanded to 8 disc).

[guest]

Update today, MKVTooNix update to 39.0.0, core 2019.11.08, and scripts 2019.11.08.
@Atak Anything that could affect us or more along the lines of bug fixes?

BIG changes in Encoding Client for server "control" :)

Can't wait to check this out :)

[mparade]

I think the first thing is to verify that you can connect to each process. From your Ripbot machine, install nmap (they have a windows version).

Then using nmap, see if the Encoding Server ports are open on your other machines. I would run command like this from Linux:
nmap -p 1000,2000,3000,4000 192.168.9.7

My results look like this:
nmap -p 1000,2000,3000,4000 192.168.9.7
Starting Nmap 7.80 ( https://nmap.org ) at 2019-11-07 05:22 MST
Nmap scan report for taffy (192.168.9.7)
Host is up (0.0023s latency).

PORT STATE SERVICE
1000/tcp open cadlock
2000/tcp filtered cisco-sccp
3000/tcp filtered ppp
4000/tcp filtered remoteanything

Nmap done: 1 IP address (1 host up) scanned in 1.25 seconds

I'm running one Encoding Server on that client. The scan shows port 1000 as open, and the other ports as filtered. Filtered will be either there is nothing listening on those ports or a firewall has filtered them.

That will at least tell you if you have some kind of network problem.

Thank you very much for the help.
I got this result:

Nmap scan report for amd-tr4.alllight.hungary (192.168.2.24)

Host is up (0.012s latency).



PORT STATE SERVICE

1000/tcp open cadlock

2000/tcp open cisco-sccp

3000/tcp open ppp

4000/tcp open remoteanything



Nmap done: 1 IP address (1 host up) scanned in 2.33 seconds

[Atak_Snajpera]

Thank you very much for the help.
I got this result:

You were supposed to do that test from work-pc to home-pc. (192.168.88.178)

[mparade]

From server machine:

Starting Nmap 7.80 ( https://nmap.org ) at 2019-11-09 18:52 Közép-európai téli ido

Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn

Nmap done: 1 IP address (0 hosts up) scanned in 3.21 seconds



screenshot about it:

http://mammutmail.com/hu/download/aa18c3bcdfc3eb71d61b3c97600352e6

[Atak_Snajpera]

Ask your admin for help. I guess that your vpn has been configured to block uncommon ports (1001,1002,1003 and so on). Weird stuff...

Next time use www.postimage.org site for images because that full screen adv is driving me crazy ;)

[mparade]

Thx! + ok.

[byteshare]

BIG changes in Encoding Client for server "control" :)

Can't wait to check this out :)
Ah, I see what you mean now. You can suspend or shutdown a server.
Not sure if anything else changed.

[Dhry]

There seems to be something wrong with the latest 1.25 of Ripbot and the 1.16 EncodingServer. I had reported issues with trying to get a batch to start properly (adding a file via Add worked, though), however as of today, even adding a file via that method did not work with distributed encoding. What happens is that the file demuxes and this and that fine, then the dist encoding master window pops up for pass1 and nothing happens. None of the server farm responds. Their encodingserver instances literally do not receive the TCP command at all. Nothing appears in the logging console. I switched out for 1.15.1.0 of the encodingserver and THEN they started to receive commands but were unable to open the file for writing to my shared folder. It's only when I went back to Ripbox 1.24 and used IT'S encoding client, and encodingserver 1.15.1.0 on all of my boxes, that everything started working again. INCLUDING batch processing! Some weird networking change has evidently been done in the code which breaks connectivity. I disabled my firewall and tested pings between the machines, no problem at all. The problem now is that using RB 1.24 I get the old error stating that video.265 file could not be opened. Are there any known distributed encoding connectivity issues that others have experienced with RB 1.25 and EncodingServer 1.16? Using Windows 7 SP1.

--Dhry

[osgZach]

Atak,

I don't recall if I've ever asked this before, although I get the feeling I may have broached the subject in years previous. I'm curious about what method you are employing to split the files (I don't see anything "obvious" in the tools directory except an executable called vjoin, which I assume is for joining). I'm trying to do some research on the best way to accomplish splitting/joining accurately and efficiently without any glitches. I've seen references online to using ffmpeg to split on keyframes (and ffprobe to acquire an index of keyframes to help with this) but haven't really found any real turnkey solution, outside of something like DVE https://github.com/nergdron/dve



slightly related, do you have any plans to ever expose ffmpeg for use by the user (as opposed to x264/x265)? I'm mainly interested in targeting and accelerating VP9 (libvpx for now) across multiple cores/machines, but down the road when Intel's SVT libraries become more mature this could be useful to a lot of people as well, especially with AV1

[Atak_Snajpera]

There seems to be something wrong with the latest 1.25 of Ripbot and the 1.16 EncodingServer. I had reported issues with trying to get a batch to start properly (adding a file via Add worked, though), however as of today, even adding a file via that method did not work with distributed encoding. What happens is that the file demuxes and this and that fine, then the dist encoding master window pops up for pass1 and nothing happens. None of the server farm responds. Their encodingserver instances literally do not receive the TCP command at all. Nothing appears in the logging console. I switched out for 1.15.1.0 of the encodingserver and THEN they started to receive commands but were unable to open the file for writing to my shared folder. It's only when I went back to Ripbox 1.24 and used IT'S encoding client, and encodingserver 1.15.1.0 on all of my boxes, that everything started working again. INCLUDING batch processing! Some weird networking change has evidently been done in the code which breaks connectivity. I disabled my firewall and tested pings between the machines, no problem at all. The problem now is that using RB 1.24 I get the old error stating that video.265 file could not be opened. Are there any known distributed encoding connectivity issues that others have experienced with RB 1.25 and EncodingServer 1.16? Using Windows 7 SP1.

--Dhry
What happens If you run ripbot264 on other machine? Do you still see the same issue? Record you desktop with some software (fraps or other) and upload somewhere. I would like to see that weird behaviour with my own eyes.

Ps. I also have win7 on all my machines and everything just works.

[guest]

There seems to be something wrong with the latest 1.25 of Ripbot and the 1.16 EncodingServer. I had reported issues with trying to get a batch to start properly (adding a file via Add worked, though), however as of today, even adding a file via that method did not work with distributed encoding. What happens is that the file demuxes and this and that fine, then the dist encoding master window pops up for pass1 and nothing happens. None of the server farm responds. Their encodingserver instances literally do not receive the TCP command at all. Nothing appears in the logging console. I switched out for 1.15.1.0 of the encodingserver and THEN they started to receive commands but were unable to open the file for writing to my shared folder. It's only when I went back to Ripbox 1.24 and used IT'S encoding client, and encodingserver 1.15.1.0 on all of my boxes, that everything started working again. INCLUDING batch processing! Some weird networking change has evidently been done in the code which breaks connectivity. I disabled my firewall and tested pings between the machines, no problem at all. The problem now is that using RB 1.24 I get the old error stating that video.265 file could not be opened. Are there any known distributed encoding connectivity issues that others have experienced with RB 1.25 and EncodingServer 1.16? Using Windows 7 SP1.

--Dhry

What happens If you run ripbot264 on other machine? Do you still see the same issue? Record you desktop with some software (fraps or other) and upload somewhere. I would like to see that weird behaviour with my own eyes.

Ps. I also have win7 on all my machines and everything just works.

I concur...

I only just got around to "testing" the latest update (08-11-19), and it's completely ruined DE !!!

Not using DE works fine !!

So I have recorded my desktop (using OBS, Fraps doesn't like W10, apparently).

https://www.mediafire.com/file/aoz3zppgbcvrbia/2019-11-13_13-49-35.rar/file

I have shown most of RB screens in the capture...there's just nothing connecting.

Also, there needs to be a "flyout" of what the new buttons do, in the left column of Encoding Client do (you'll see what I mean in the capture)

Also, running W10 1903, and just because it works on W7, doesn't mean it will work on other versions...

You might also notice that when the Job is aborted, it kills RB, but leaves the Clients in the Task Bar.

I have had to return to the previous build.

And another thing I'd like to mention (which has been a problem for me for a long time), after you add a new Job, and then start that Job, it won't start processing....RB has to be re-started beforehand !!

[MCFish]

Same here. None of the encoding servers get any connection with latest update.

[ReinerSchweinlin]

I noticed one new thing:
After changing my Numa Settings in my proliant D380G7 (2 x Xeon X5650) to not use NUMA but instead treat the two CPUs as one - I get almost 96% CPU Usage when using one encodingserver with x265 slow settings - it is blazing fast now. Bevore, I had to use 4 servers to max out CPU (only de-interlacing used, nothing else).

Did anything change in recent x265 you use?

[Ryushin]

I'm getting a error on the DE Server:
x265 [Error]: main10 profile not compatible with i444 input chroma subsampling

Brought a fresh job in of a 4K HDR mkv file, that Ripbot made from the original 4K source a few days ago. Changed the size down to 720p. Set 5.1 AAC, 21 CRF. X265 with the default options plus sar 1:1 (not sure why sar 1:1 is not the default as I've been bitten by stretching in 4x3 shows). Server Chunk fails with that error immediately as it tries to start it.

Also, older 4K jobs that I brought in from disc source from a couple of weeks ago just show "Starting...." and then nothing ever starts. Encoding client shows processor usage about 15%, but nothing ever starts.

Edit: Well, one of the older jobs eventually started running after about 30 minutes. However, the job before it had Starting... for six hours and never did anything.

[Atak_Snajpera]

I concur...

I only just got around to "testing" the latest update (08-11-19), and it's completely ruined DE !!!

Not using DE works fine !!

So I have recorded my desktop (using OBS, Fraps doesn't like W10, apparently).

https://www.mediafire.com/file/aoz3zppgbcvrbia/2019-11-13_13-49-35.rar/file

I have shown most of RB screens in the capture...there's just nothing connecting.

Also, there needs to be a "flyout" of what the new buttons do, in the left column of Encoding Client do (you'll see what I mean in the capture)

Also, running W10 1903, and just because it works on W7, doesn't mean it will work on other versions...

You might also notice that when the Job is aborted, it kills RB, but leaves the Clients in the Task Bar.

I have had to return to the previous build.

And another thing I'd like to mention (which has been a problem for me for a long time), after you add a new Job, and then start that Job, it won't start processing....RB has to be re-started beforehand !!

Win10 1909
https://i.postimg.cc/hG4r4968/Untitled-1.png

[byteshare]

I noticed one new thing:
After changing my Numa Settings in my proliant D380G7 (2 x Xeon X5650) to not use NUMA but instead treat the two CPUs as one - I get almost 96% CPU Usage when using one encodingserver with x265 slow settings - it is blazing fast now. Bevore, I had to use 4 servers to max out CPU (only de-interlacing used, nothing else).

Did anything change in recent x265 you use?
Not the most recent update but maybe 2 versions ago (no version numbers for the changes so I can't recall)? There were some changes to RB and I saw the same effect of not needing as many servers to max out CPU. Had something to do with the multiple processing threads but I'm not sure exactly what was changed or what was the issue before hand.
For my encoding machines on Core version: 2019.10.15
I have a test machine for the latest builds and once they work how I like I change to a newer version.

[guest]

Win10 1909
https://i.postimg.cc/hG4r4968/Untitled-1.png

So, we're just posting screen shots now, are we ???

My crystal ball musn't be working today....

So this is either you showing that it works, or you've fixed it, or you're just rubbing our nose's in it !!!

I got a new update this morning, only "script's", updated the pc that I did that screen recording on yesterday, loaded a new job, and exactly the same scenario....NOTHING, no TCP connection !!!

Ran build 22-10-19, loaded the same job, started that job, and away she goes, not problem, on the same pc.

What's changed ???

PS:- Just tried it on another W10 (1909) pc, and the same result, further comments in my reply to Dhry.

POORLY DONE !!!!!

[Dhry]

The screenshot seems to show that RB works with Win10 only. I'm pretty sure something's broken with the tcp connectivity on Win7 at least. It's literally not making a outbound connection at all.
I'll go back to the newest versions and try again.
Edit: it isn't updating. Could the first post in this thread be changed to link to 1.25 of RB please? Still linked to 1.24.

[guest]

The screenshot seems to show that RB works with Win10 only. I'm pretty sure something's broken with the tcp connectivity on Win7 at least. It's literally not making a outbound connection at all.
I'll go back to the newest versions and try again.
Edit: it isn't updating. Could the first post in this thread be changed to link to 1.25 of RB please? Still linked to 1.24.

Thanks Dhry, I'm glad I'm not the only one pissed off about this.

I have just tested it on 2 different W10 (1909) pc's, with the same result...NOTHING, it's f*****.

I'm not sure what the idea of that screenshot is, with NO explanation.

The only way I can get RB to do anything after that last 2 updates is to NOT use DE, OR go back to the previous build (22-10-19).

[ReinerSchweinlin]

Not the most recent update but maybe 2 versions ago (no version numbers for the changes so I can't recall)? There were some changes to RB and I saw the same effect of not needing as many servers to max out CPU. Had something to do with the multiple processing threads but I'm not sure exactly what was changed or what was the issue before hand.
For my encoding machines on Core version: 2019.10.15
I have a test machine for the latest builds and once they work how I like I change to a newer version.
I noticed the CTU size smaller then usual when using RB with x265, which seems to help with many cores, maybe something changed here, too..

[guest]

I noticed one new thing:
After changing my Numa Settings in my proliant D380G7 (2 x Xeon X5650) to not use NUMA but instead treat the two CPUs as one - I get almost 96% CPU Usage when using one encodingserver with x265 slow settings - it is blazing fast now. Before, I had to use 4 servers to max out CPU (only de-interlacing used, nothing else).

Did anything change in recent x265 you use?

So you're saying that NUMA isn't an advantage, anymore ??

[Atak_Snajpera]

The screenshot seems to show that RB works with Win10 only. I'm pretty sure something's broken with the tcp connectivity on Win7 at least. It's literally not making a outbound connection at all.
I'll go back to the newest versions and try again.
Edit: it isn't updating. Could the first post in this thread be changed to link to 1.25 of RB please? Still linked to 1.24.

Like I said before. Everything just works
https://i.postimg.cc/kqc6K7VC/Untitled-1.png

[guest]

Like I said before. Everything just works

Godammit, so please explain to me why the previous build works, and then it gets updated with this mess, and it's useless !?!?!?

Something in the fairly major change to Encoding Client, has impacted the TCP connection !!

So you've clearly shown that it works for you on both W7 & W10, but you have to advantage over us, YOU know what you've changed, and we don't.

Did you view that desktop capture I posted ??? what else do you need ??

[Atak_Snajpera]

My theory is that your firewall Has detected that checksum for encodingclient.exe/encodingserver.exe changed and by default blocks again all outgoing connections.

[guest]

My theory is that your firewall Has detected that checksum for encodingclient.exe/encodingserver.exe changed and by default blocks again all outgoing connections.

Well, I doubt that's the problem, as I have ALL the Firewall's disabled.

But why would that change between RB builds ??

[Atak_Snajpera]

Use nmap to check If ports 1000,2000 and so on are open.

[guest]

Use nmap to check If ports 1000,2000 and so on are open.

I will get Nmap, and do what it does, and I will get back to you.

Nevertheless, you must have changed something, in the last update !!!

Update:- got Nmap, don't know what I have to do.....

[ReinerSchweinlin]

So you're saying that NUMA isn't an advantage, anymore ??
Well, I can´t be sure this is valid for every dual CPU configuration. I switched NUMA off because I can´t get any grafix card to run in my DL380, so I am using the CPU for OPENCL (which is slow, but in some of my cases is better than nothing). This way all of the cores are used for OPENCL Filtering, which gave quite a bosst.
I then tested Encodes without OPENCL usage and encountered the CPU Usage to bee much higher than before when using only one encodingserver. Same time some updates came from RB Update Server - so I can´t be sure what´s exactly the cause of this right now, would have to switch back to NUMA mode and compare. Didn´t have time for that, that´s because I was asking here, if something had changed within x265/RB (after all, maxing out more cores while encoding can be usefull for bigger multicore CPUs as well).

[guest]

Well, I can´t be sure this is valid for every dual CPU configuration. I switched NUMA off because I can´t get any grafix card to run in my DL380, so I am using the CPU for OPENCL (which is slow, but in some of my cases is better than nothing). This way all of the cores are used for OPENCL Filtering, which gave quite a bosst.
I then tested Encodes without OPENCL usage and encountered the CPU Usage to bee much higher than before when using only one encodingserver. Same time some updates came from RB Update Server - so I can´t be sure what´s exactly the cause of this right now, would have to switch back to NUMA mode and compare. Didn´t have time for that, that´s because I was asking here, if something had changed within x265/RB (after all, maxing out more cores while encoding can be usefull for bigger multicore CPUs as well).

OK, aside, have you got the very latest RB update ???

If so, are you have DE problems ??

[Atak_Snajpera]

https://forum.doom9.org/showthread.php?p=1889796#post1889796

[guest]

https://forum.doom9.org/showthread.php?p=1889796#post1889796

:thanks:

OK, did a simple scan on the laptop that has the latest RB on it, and here's the results...

Starting Nmap 7.80 ( https://nmap.org ) at 2019-11-15 00:01

Nmap scan report for DELL-XPS-M1330.gateway (10.0.0.132)

Host is up (0.00s latency).


PORT STATE SERVICE

1000/tcp closed cadlock
2000/tcp closed cisco-sccp
3000/tcp closed ppp
4000/tcp closed remoteanything
5000/tcp closed upnp
6000/tcp closed X11
7000/tcp closed afs3-fileserver
8000/tcp closed http-alt

Nmap done: 1 IP address (1 host up) scanned in 0.78 seconds

So what does this mean ??

I will run some test on the one of my "proper" RB pc's tomorrow.

[Atak_Snajpera]

This is how it should look...
https://i.postimg.cc/Z5mFdKf2/Capture.png
All your ports are closed. Check anti-virus or other anti-malware software.

[Privateer5000]

Same Problem here with DE Mode not working on 2 machines. Used nmap and it says STATE open like in the picture above. Tried it on Win10 1903, just upgraded to 1909 but still nothing happens.

[guest]

This is how it should look...
https://i.postimg.cc/Z5mFdKf2/Capture.png
All your ports are closed. Check anti-virus or other anti-malware software.

Only using built in Windows Defender...

But it still MUST be something you've changed !!!!

[Atak_Snajpera]

But it still MUST be something you've changed !!!!
Yes I have added...
if PaulyDunnePC=true then BlockDEmode

[guest]

Yes I have added...
if PaulyDunnePC=true then BlockDEmode

Seriously...is that the best you can do !!

Its like beating your head against a wall, why does the previous build work, and the latest one doesn't ??

It's not just me !!!

[Donholio]

Hi, everyone!

I just want to calmly let everyone know that my corporate antivirus found an issue in the waitforprocess.exe file that is downloaded by the updater for RipBot264. The file is hosted here: http://atak-snajpera.5v.pl/ripbot264update/WaitForProcess.zip according to the update.ini file.

This is the entry from Cisco Umbrella:

o Event Type: Executed malware
o Computer: <REDACTED>
o Hostname: <REDACTED>
o IP: <REDACTED>
o Detection: W32.31A417AE41-85.LP.RET.SBX.TG
o File: WaitForProcess.exe
o File path: file:///<REDACTED>RipBot264v1.24.0/tools/WaitForProcess/WaitForProcess.exe
o Detection SHA-256: 31a417ae412e9e64799565f00ff1efbcd665caf2b8433e63af0a9c59653eaaf7
o Application SHA-256: 3685495d051137b1c4efde22c26df0883614b6453b762fa84588da55ed2e7744
o Severity: High
o Timestamp: 2019-11-14 12:26:18 +0000 UTC

The SHA-256 for the application shows very little detection information. When I run the Application SHA through Virustotal's search function, it shows clean. When I run the Detection SHA (the part of the file detected as "malware"), it shows detections by CrowdStrike Falcon, Panda, Microsoft, Trapmine, and Webroot. Anyone can check this themselves by copying the SHA256 values above, logging into VirusTotal, and clicking the Search function.

It looks like the detected malware might be adware-related or part of an SEO inflation scheme. It also looks like there might be a trojan that modifies windows firewall settings.

I want to scan the actual file through VirusTotal, but unfortunately, the application is blocked by Cisco Umbrella. I will test my home machine once I get home, but if the SHA values match, there should be no different information.

I love RipBot264 and want to continue using it. I've updated the update.ini and installed_tools.ini files in the Tools directory and the update.ini file in the Updates directory to set waitforprocess's AllowUpdate value to 0 for now. I'm not sure what the application is used for at this point. I'd be interested to know what I'm breaking by not having it around.

I think what I've found is credible and I feel it would be best to get this information out there ASAP. Please let me know if this is a known issue and a false positive.

Like I said to start, I'm not trying to cause a panic here and I'm a huge fan of the program.

Thanks

-Donholio

[Atak_Snajpera]

WaitForProcess.exe is just a helping tool I wrote to spawn multiple executables and aggregate percentage progresses into single percentage progress (which is then displayed in ripbot264). It is used during indexing and encoding audio. Some anti-viruses are VERY allergic to executables compressed by UPX. ( https://upx.github.io/ )
If you delete WaitForProcess.exe then indexing and audio encoding won't work.
https://i.postimg.cc/Zq4JTWh5/Untitled-1.png

It is obvious that it is false positive detection...
https://www.virustotal.com/gui/file/31a417ae412e9e64799565f00ff1efbcd665caf2b8433e63af0a9c59653eaaf7/detection

[byteshare]

Only using built in Windows Defender...
But it still MUST be something you've changed !!!!
Do you have exceptions for the RB folder and temp folders?

Same Problem here with DE Mode not working on 2 machines. Used nmap and it says STATE open like in the picture above. Tried it on Win10 1903, just upgraded to 1909 but still nothing happens.
Do you have Windows Defender with exceptions for RB and Temp Folder?
Does an older version work for you like Pauly?

On Core 2019.11.08 and Encoding Server 1.16.0.0 I have RB working in DE mode with two different Windows 10 computers, both 1903 (one Pro and one Home)

Yes I have added...
if PaulyDunnePC=true then BlockDEmode
I found this funny. :goodpost:

[guest]

Do you have exceptions for the RB folder and temp folders?


Do you have Windows Defender with exceptions for RB and Temp Folder?
Does an older version work for you like Pauly?

On Core 2019.11.08 and Encoding Server 1.16.0.0 I have RB working in DE mode with two different Windows 10 computers, both 1903 (one Pro and one Home)


I found this funny. :goodpost:

No, I don't have any exceptions....I've never had to before !!!

Why can't he just say that's he's changed something...

Just a sick IT sense of humour...it wouldn't have worked, anyway..

[Atak_Snajpera]

My last attempt to help you. Create virtual machine and then install fresh win10 1909. See If you can connect to servers within virtual machine. Virtual win10 also works on my pc. If still nothing then i give up.

[byteshare]

No, I don't have any exceptions....I've never had to before !!!

Why can't he just say that's he's changed something...

Just a sick IT sense of humour...it wouldn't have worked, anyway..
I had to add the exceptions and have had them for a while, so not sure what is causing your issue since your port is open.
RB is working on 4 Win10 machines for me right now after testing the newest version. I have Windows Fire Wall on, but with exceptions for RB and Windows Defender exceptions for RB.

[ReinerSchweinlin]

OK, aside, have you got the very latest RB update ???
Yes, at least the one from this morning which was the last time RB was started on all machines.


If so, are you have DE problems ??
No. Mixed WIN 10 and WIn 7 PCs, some over VPN, different CPUs/GPUs...

[guest]

My last attempt to help you. Create virtual machine and then install fresh win10 1909. See If you can connect to servers within virtual machine. Virtual win10 also works on my pc. If still nothing then i give up.

This is your software, and your update, so you need to help until this is resolved.

Not interested in VM nonsense, I just installed W10 1909 on another pc, left it all default, vanilla settings, installed what RB needs to work, it ask to do the exceptions it needed, loaded a new job, and exactly the same non starting, non connecting behaviour.

So I need some one to tell me exactly what they did, to get this latest build working in DE !!

I still don't understand why a working app, stops working after an update, and the creator of that update can't or won't tell us what he's done, or how to fix it.

I'll ask you again, did you view that screen capture I posted ???

That's what happens, just a lot of little boxes & icons flashing, not doing ANYTHING, no info in the TCP window..it's just dead !!!

So for now, I will just have to run with the previous working build, and turn off auto updates.

[howzz]

i noticed that with the latest update, "gathering information" is once again taking a lonnnnng time. except this time, regardless of choosing Lmash, FFmp2 or GPU makes it any faster. i noticed the update log says the script was updated. just exactly what changed made Gathering Information so much slower now.

i also noticed that it takes forever for the encode to begin now. from the time i click "start", to seeing all the cores loaded, it takes like 2 mins. before it used to be instant.

[guest]

i noticed that with the latest update, "gathering information" is once again taking a lonnnnng time. except this time, regardless of choosing Lmash, FFmp2 or GPU makes it any faster. i noticed the update log says the script was updated. just exactly what changed made Gathering Information so much slower now.

i also noticed that it takes forever for the encode to begin now. from the time i click "start", to seeing all the cores loaded, it takes like 2 mins. before it used to be instant.

Join the club, mate...

I can't DE to work at all, since last update, consider yourself lucky that it only take 2 minutes.

[Atak_Snajpera]

This is your software, and your update, so you need to help until this is resolved.

Not interested in VM nonsense, I just installed W10 1909 on another pc, left it all default, vanilla settings, installed what RB needs to work, it ask to do the exceptions it needed, loaded a new job, and exactly the same non starting, non connecting behaviour.

So I need some one to tell me exactly what they did, to get this latest build working in DE !!

I still don't understand why a working app, stops working after an update, and the creator of that update can't or won't tell us what he's done, or how to fix it.

I'll ask you again, did you view that screen capture I posted ???

That's what happens, just a lot of little boxes & icons flashing, not doing ANYTHING, no info in the TCP window..it's just dead !!!

So for now, I will just have to run with the previous working build, and turn off auto updates.

Stop using this free software. I owe you nothing. I'm done with you.

[slalom]

Seriously...is that the best you can do !!

Its like beating your head against a wall, why does the previous build work, and the latest one doesn't ??

It's not just me !!!
Download the file from first page. Run it and update
Replace your files except *.ini

open RipBot and run your jobs

[guest]

Download the file from first page. Run it and update
Replace your files except *.ini

open RipBot and run your jobs

Hi slalom,

I have actually thought of doing that....I have tried that a few months ago, when there was a weird problem.

Thanks for the tip :)

[guest]

Stop using this free software. I owe you nothing. I'm done with you.

Why am I not surprised by your response...

I know you probably won't reply to this, but there are some things I want to say, like it or not.

You're obviously a pretty intelligent person, I have been trying to figure out for years now, whether you're some young smart-arse, or a grumpy old man (I think the former), I doubt that an older person would post a reply/comment like that....but then what would I know, I'm just a grumpy old man.

All I was trying to do was help you diagnose a major problem with your "free" software, that other people were commenting on, I sent you a video capture (of which you asked someone else to provide) of the strange behaviour, on my setup, and all you could do was rub it in my face, by posting screenshots of it working on your setup, with NO explanation !!!

I also asked several times for an explanation on what you had changed in the auto update that ruined everything, and you simply didn't....you asked me to get specific app's to check things that I have no idea about what I'm doing, or what I'm looking for.

You seem to expect everyone out there to have the same level of expertise as you...well, we don't !!!

I gave you as much info as I could, but you simply didn't want to explain anything, all you did was say, "It just work's".

It has to be something to do with the changes you added for server control.

Are you going to help the others that are starting to comment on the same error's, are you going to ignore them, and tell them, "I'm done with you" ???

You can't stop me from using this software, I know some day, in hopefully the not too distant future, you WILL fix it, I WILL get the update's, and I'll be happy.

I WILL keep perusing this forum, and if I feel the need, I WILL post a comment.

Like I just did....

[byteshare]

Why am I not surprised by your response...
I've mentioned it before a while back. It isn't that you're asking for help. It is how you're asking for help (in my opinion). This is free software, so you can only expect limited support--if any. Unless the issue you're having is experienced by everyone it is possible to not get support, which has happened to me as well, but often it is something on my end that I eventually figure out.

[byteshare]

Yes, at least the one from this morning which was the last time RB was started on all machines..
Latest I can see is Core 2019.11.08, Scripts 2019.11.13, MKVtoolnix 40.0.0
(would be easier to tell if the point versions changed with each update)

i noticed that with the latest update, "gathering information" is once again taking a lonnnnng time. except this time, regardless of choosing Lmash, FFmp2 or GPU makes it any faster. i noticed the update log says the script was updated. just exactly what changed made Gathering Information so much slower now.

i also noticed that it takes forever for the encode to begin now. from the time i click "start", to seeing all the cores loaded, it takes like 2 mins. before it used to be instant.
As far as I understand it, no matter what the default decoder is you're still doing LSmash when adding a job so that if you want to switch decoders later it isn't super slow, which is why if you look in the job folder you'll still see a *.lwi file, also when starting a job it does the compression and such for being able to send the LSmash file (even if it isn't going to be used). So...I believe it is slow for you again because of something to do with LSmash.
--I could be wrong about the above, but that is how I understood it.--
I'm not sure who is switching their decoder all of the time but personally I would prefer if only the selected default decoder is run when adding a job to not waste CPU/electricity/time--adds up over time ;)

[slalom]

Hi slalom,

I have actually thought of doing that....I have tried that a few months ago, when there was a weird problem.

Thanks for the tip :)
Did you try it?
That solved my problem with LSMash some time ago

[Privateer5000]

Sry for the late response, but i got a really busy day. @byteshare i also use only Windows Defender, i already had an exception for the Temp Directory but not for the Ripbot Directory. I also excluded the Ripbot folder but it changed nothing. After Going back to Version 1.24 from the first post here DE Mode was working again but as soon as i allow Ripbot updating itself to 1.25 DE Mode is broken again. There is absolutely no TCP communcation between Encoding Server and Client.

[guest]

Sry for the late response, but i got a really busy day. @byteshare i also use only Windows Defender, i already had an exception for the Temp Directory but not for the Ripbot Directory. I also excluded the Ripbot folder but it changed nothing. After Going back to Version 1.24 from the first post here DE Mode was working again but as soon as i allow Ripbot updating itself to 1.25 DE Mode is broken again. There is absolutely no TCP communcation between Encoding Server and Client.

Hi Privateer5000,

I'm so happy that you've got the same problem I have....I have tried to supply as much info I can to the dev, and he's not responded well, in fact if you go back a couple of posts, you'll see what he said to me.

Because it works for him, he doesn't see a problem.

BTW, posting attachments on this forum isn't a very good idea, takes way too long for them to be approved.

[guest]

Latest I can see is Core 2019.11.08, Scripts 2019.11.13, MKVtoolnix 40.0.0
(would be easier to tell if the point versions changed with each update)

I got an update this morning, only got mkvtoolnix v40, lsmash had an error, and couldn't be downloaded...

No changes to core...

Nice subtle hint there too, btw (doesn't work tho)

[guest]

Did you try it?
That solved my problem with LSMash some time ago

Yes, I have now tried it, and unfortunately, it didn't worked :(

[byteshare]

Sry for the late response, but i got a really busy day. @byteshare i also use only Windows Defender, i already had an exception for the Temp Directory but not for the Ripbot Directory. I also excluded the Ripbot folder but it changed nothing. After Going back to Version 1.24 from the first post here DE Mode was working again but as soon as i allow Ripbot updating itself to 1.25 DE Mode is broken again. There is absolutely no TCP communcation between Encoding Server and Client.
Hmm, I'm honestly not sure what has changed since I'm not having an issue, and you have already checked the Windows Defender and Firewall exceptions. And, if I remember correctly, confirmed your port is open.
Also, Pauly Dunne already mentioned better to link to an image from an image hosting site such as Imgur.
So, I'll try to think of something but sorry I'm out of ideas at the moment.
If there was a way to control which version you're on at least then you could roll back to a specific version that worked for you. I keep backups for this reason.

[howzz]

speaking of version control. is there a way to get to a previous version? i noticed the main page only gives you up to v24, but there's been countless updates since v24. i just need the version prior to Nov 3rd.

[guest]

speaking of version control. is there a way to get to a previous version? i noticed the main page only gives you up to v24, but there's been countless updates since v24. i just need the version prior to Nov 3rd.

It would be nice to have that...

Na, you're plum outta luck on that one, 'cause I'm sure you know, if you were to download v1.24 off the 1st page, and then do an update, you'd be right back with the problem build.

So you'll be after core build 22-10-18, the last one that DE works with ?

However, having said that, I might be able to help you with your problem..

PM me...

[guest]

In my quest to try and figure what might be the problem with DE not working for several of us, I have been swapping & changing different versions of EncodingClient.exe & EncodingServer.exe, using RB build (22-10-19), I know I'm no wiz when it comes to this, but I'm thinking that it might be the Encoding Client that has a problem, as from what I can see, it's looking for the wrong ports.

Have a look at the 2 screenshots, and decipher it for yourself..

using newest server, with older client.

https://www.mediafire.com/view/42ijlyrc5weqcna/wrong_ports.jpg/file

compared to this...what it should look like..

https://www.mediafire.com/view/6bt2apknpcjoos4/this_works.jpg/file

I'd guess that it's not desirable to mix and match, but you've gotta try a few things...trial & error, and I could be completely off the mark, here.

[byteshare]

In my quest to try and figure what might be the problem with DE not working for several of us, I have been swapping & changing different versions of EncodingClient.exe & EncodingServer.exe, using RB build (22-10-19), I know I'm no wiz when it comes to this, but I'm thinking that it might be the Encoding Client that has a problem, as from what I can see, it's looking for the wrong ports.

Have a look at the 2 screenshots, and decipher it for yourself..

using newest server, with older client.

https://www.mediafire.com/view/42ijlyrc5weqcna/wrong_ports.jpg/file

compared to this...what it should look like..

https://www.mediafire.com/view/6bt2apknpcjoos4/this_works.jpg/file

I'd guess that it's not desirable to mix and match, but you've gotta try a few things...trial & error, and I could be completely off the mark, here.
Guessing that the old Encoding Server 1.15.2.0 didn't work with the newer Encoding client?
also, what is happening on the "Main" window of the Encoding Server? If you can make a really good step-by-step of the things you've tried and the issues, maybe, someone would be able to figure it out. Not saying I'll figure it out, but you never know...sometimes I'm even right twice in a day (broken clock).

[guest]

Guessing that the old Encoding Server 1.15.2.0 didn't work with the newer Encoding client?
also, what is happening on the "Main" window of the Encoding Server? If you can make a really good step-by-step of the things you've tried and the issues, maybe, someone would be able to figure it out. Not saying I'll figure it out, but you never know...sometimes I'm even right twice in a day (broken clock).

Hey byteshare...

yes I know I probably should have written it down step by step, I also would think that mixing up different versions and build stuff probably isn't a good idea, but I was just a bit puzzled about incorrect looking port numbers, etc.

Maybe I'll do it again, and this time I'll document it.

edit:- or maybe I won't, with more comments on a problem, surely it will be addressed soon.

[Privateer5000]

I am not quite sure concerning the Firewall exceptions, will check it later the day. But Windows Defender has exceptions for both, Ripbot and Ripbot Temp Folder. Do i have to add exceptions to my Router Firewall as Well or only in NY machine? I mean at the moment the Client does not reach the Encoding Server running in the same machine. Another thing i realized but i do not know If it ist Important. With Version 1.24 in thhe Overview Window the Encoding Server ist named after my machine's Networkname and with Version 1.25 it ist still named "Server 1". Is it possible that the Main Problem ist that the Client is not able to detect the Encoding Server? And yes, since Yesterday in the evening i cannot create Jobs anymore, Ripbot stops with the Message that a DLL from LSmash is Missing.

[MCFish]

Just tried in a clean vm, and same happens. Queue in chunk status and no progress

[Privateer5000]

So, checked Firewall Setting, has exceptions for Ripbot Encodingclient and Encodingserver, for testing i deactivated the firewall completly but no change. Chunks are being prepared and then nothing. The Client tries to connect to the Server but the Server do not recognize anything.

[Ronski]

I seem to be having the same problem, I've not used Ripbot for a while, went to use it today, both installations updated, now it just sits there with the encoding client showing "Pass1...[0.0%] 0/163714 frames, 0.00 fps, 0 kbps, eta" and the encoding servers showing waiting for connection on both machines. The encoding client shows all my encoding clients as offline.

Just noticed on my main PC the servers are v1.15.2.0 and on the other it's v1.16.0.0.

If I set distributed encoding to auto reconnect then the server icons in the client flicker as it seems to try to reconnect.

[FuzzyNutz]

I identified this issue here before and it seemed to be addressed shortly after. Now the issue is back. If the version of MKVToolNix GUI on a users system is newer than that associated with the RipBot264 update, the update should leave MKVToolNix untouched.

Please make it so!

[guest]

I seem to be having the same problem, I've not used Ripbot for a while, went to use it today, both installations updated, now it just sits there with the encoding client showing "Pass1...[0.0%] 0/163714 frames, 0.00 fps, 0 kbps, eta" and the encoding servers showing waiting for connection on both machines. The encoding client shows all my encoding clients as offline.

Just noticed on my main PC the servers are v1.15.2.0 and on the other it's v1.16.0.0.

If I set distributed encoding to auto reconnect then the server icons in the client flicker as it seems to try to reconnect.

Just like this :-

http://www.mediafire.com/file/aoz3zppgbcvrbia/2019-11-13_13-49-35.rar/file

[guest]

Hey guys,

There was another auto update posted "overnight" (my time), for core & lsmash (lsmash from failed download yesterday).

Only ripbot264.exe was updated, however, sadly the DE problems persists :(

Better luck next time...

[Dhry]

I had to return to RB 1.24 (core 20190123) with encodingclient 1.12.10 and encodingserver 1.15.2 on the machines in my farm. It works. I have to run the mux.cmd etc stuff manually for every file I batch encode, but it works.
Will stay tuned to this thread in case the fix arrives. But one thing's for absolute certain, the latest RB has significant issues with DE. Whatever connectivity-oriented code that was updated in EC 1.16 pretty much needs to be rolled back. Thanks for your work Atak.

Dhry

[guest]

You might be done with me, but I'm done, here...

I have a suggestion for future updates, that should prevent situations like this, where more & more users are commenting on the DE problem, that I tried to help with.

There is very little control over how updates for RB work, you either get them, or you don't...simple as that.

The problem is, it all happens quite covertly in the background (if you're online), and the only noticable way you know there's been an update is the next time you start the app, and you see the little pop up window appear as the updates are being installed, and by that time it's too late to do anything about it.

Then, if there's a major problem (like now), the user has very limited options, either try to use the app as is, OR download the latest available FULL version from page 1, and leave it at that, as doing an autoupdate just takes you straight back to the problem build (vicious circle).

Unfortunately, the only available build is v1.24, which is way back in January '19, as you haven't bothered to post v1.25, for what ever reason, not that that would be much different, in this case, but better.

So my suggestion is, that you implement some "pop up" that tells the user that "an update is available, would you like to download it", that gives the user a heads up, and a choice, and gives them time to maybe backup their current configuration, so just incase theres a problem, they can be roll it back to the previous working state.

And another thing that might help, is that EVERY update has an incremental change to the version number...we've been on 1.25.0 for months, it should be 1.25.34, or 1.30 by now....

[byteshare]

You might be done with me, but I'm done, here...

I have a suggestion for future updates, that should prevent situations like this, where more & more users are commenting on the DE problem, that I tried to help with.

There is very little control over how updates for RB work, you either get them, or you don't...simple as that.

The problem is, it all happens quite covertly in the background (if you're online), and the only noticable way you know there's been an update is the next time you start the app, and you see the little pop up window appear as the updates are being installed, and by that time it's too late to do anything about it.

Then, if there's a major problem (like now), the user has very limited options, either try to use the app as is, OR download the latest available FULL version from page 1, and leave it at that, as doing an autoupdate just takes you straight back to the problem build (vicious circle).

Unfortunately, the only available build is v1.24, which is way back in January '19, as you haven't bothered to post v1.25, for what ever reason, not that that would be much different, in this case, but better.

So my suggestion is, that you implement some "pop up" that tells the user that "an update is available, would you like to download it", that gives the user a heads up, and a choice, and gives them time to maybe backup their current configuration, so just incase theres a problem, they can be roll it back to the previous working state.

And another thing that might help, is that EVERY update has an incremental change to the version number...we've been on 1.25.0 for months, it should be 1.25.34, or 1.30 by now....

I could see this being more user friendly, but you can disable auto update as it is to avoid your worries.

For anyone having issues with DE mode with the updated version try this one: https://mega.nz/#!loRCjSAQ!9TaUwv2yOPkBnN3M63C_ThEbON_jgyzNyONObQInsbQ
This is the first version since the change to allow suspend on a remote server. It works for me, and FYI, between this version and the one before it I was prompted by my Firewall to allow it.
If it doesn't work, I can upload the version just prior to it for testing.
With Lsmash 2019.11.16, Scripts 2019.11.13, MKVtoolnix 40.0.0, and Core 2019.11.16
It also works for me, and I believe those are the most recent versions.

[guest]

I could see this being more user friendly, but you can disable auto update as it is to avoid your worries.

For anyone having issues with DE mode with the updated version try this one: https://mega.nz/#!loRCjSAQ!9TaUwv2yOPkBnN3M63C_ThEbON_jgyzNyONObQInsbQ
This is the first version since the change to allow suspend on a remote server. It works for me, and FYI, between this version and the one before it I was prompted by my Firewall to allow it.
If it doesn't work, I can upload the version just prior to it for testing.
With Lsmash 2019.11.16, Scripts 2019.11.13, MKVtoolnix 40.0.0, and Core 2019.11.16
It also works for me, and I believe those are the most recent versions.

Hey byteshare,

Just noticed your post...

I know that you can disable auto update, but then you could fall a long way behind with updates...it would be nice if it wasn't "sneaky". lol.

I will download your "version" and see what it does, but the last one that DE work for me, was core 22-10-19, EC 1.15.2.

I was going to post the pre core 08-11-19, EC 1.16.0 version for people to fall back on.

edit: just tried your build, still doesn't work for me :(

[byteshare]

I will download your "version" and see what it does, but the last one that DE work for me, was core 22-10-19, EC 1.15.2.

I was going to post the pre core 08-11-19, EC 1.16.0 version for people to fall back on.
Yeah, you could post that.
I'm not sure what is causing issues for people because it works for me.
You could try running RipBot from a different folder name, and if you have a firewall or any thing else setup it should treat RB like it is a new program and prompt you to allow traffic and/or maybe if something else is blocking RB than that other thing blocking RB wont? Not sure, just trying to put some ideas out there.
I'm using FFMS2 as the decoder, but I think you and several other people are not using Lsmash as the default decoder for most things.
edit: just tried your build, still doesn't work for me :(
Okay, what ever changed it started with that version then for the people that are having issues with DE.
Does non-DE mode work?

[guest]

Yeah, you could post that.
I'm not sure what is causing issues for people because it works for me.
You could try running RipBot from a different folder name, and if you have a firewall or any thing else setup it should treat RB like it is a new program and prompt you to allow traffic and/or maybe if something else is blocking RB than that other thing blocking RB wont? Not sure, just trying to put some ideas out there.
I'm using FFMS2 as the decoder, but I think you and several other people are not using Lsmash as the default decoder for most things.

Isn't that almost a contradiction...I am using Lsmash, but that won't have any affect on DE.

I've got all the firewalls turned off...

Okay, what ever changed it started with that version then for the people that are having issues with DE.
Does non-DE mode work?

Haven't had any problems when not using DE !!

I don't really know why we're stuffing around with this, its clearly a problem that has to be resolved by the dev.

[Privateer5000]

@byteshare
Non-DE Mode is working for me with every Version, just DE Mode is broken.
What Firewall are you using? But shouldn't the Test with nmap make sure that the Firewall lets Traffic through?
I do not think that the choice of the Decoder ist causing the Problem because we never get that far that it is used. I will try your Version later that das, also your Suggestion to use a new Folder, maybe IT helps.

But there is something that must be said. @Pauly Dunne: you have eine good points and in some points i totally agree with you. Yes, IT would have been nice to get more Info about what was changed or even get es little hint If our testing ist helpfull or points in the wrong direction. Also your Idea with the PopUp Windows with an Update Notification ist good, wanted to make a Feature request myself after the actual Problem is solved. But i totally disagree with the way you are posting. Like Atak Said, Ripbot ist a free Software, i guess He ist making it Out of Passion and in his free time, because i do not believe he can live from the donations He may or may Not receive. No one of us has any right to ask for or request something. We all can be thankful that he shares his work with us instead of doing it only for himself. So please Stop your childish Kindergarten-like way of Posting ("better Lück next time"), it is not helpful. I would totally understand it when Atak Stop the Public support for Ripbot If everybody treats him Like you did.
Like i said, you have good ideas that i support, but not the way how you post here

[guest]

@byteshare
Non-DE Mode is working for me with every Version, just DE Mode is broken.
What Firewall are you using? But shouldn't the Test with nmap make sure that the Firewall lets Traffic through?
I do not think that the choice of the Decoder ist causing the Problem because we never get that far that it is used. I will try your Version later that das, also your Suggestion to use a new Folder, maybe IT helps.

But there is something that must be said. @Pauly Dunne: you have eine good points and in some points i totally agree with you. Yes, IT would have been nice to get more Info about what was changed or even get es little hint If our testing ist helpfull or points in the wrong direction. Also your Idea with the PopUp Windows with an Update Notification ist good, wanted to make a Feature request myself after the actual Problem is solved. But i totally disagree with the way you are posting. Like Atak Said, Ripbot ist a free Software, i guess He ist making it Out of Passion and in his free time, because i do not believe he can live from the donations He may or may Not receive. No one of us has any right to ask for or request something. We all can be thankful that he shares his work with us instead of doing it only for himself. So please Stop your childish Kindergarten-like way of Posting ("better Lück next time"), it is not helpful. I would totally understand it when Atak Stop the Public support for Ripbot If everybody treats him Like you did.
Like i said, you have good ideas that i support, but not the way how you post here

Fair enough Privateer5000, I will take your comments into consideration.

However, I don't really think it's childish to say "better luck next time", when you're waiting for something, and it doesn't happen.

[slalom]

I read about your problems
My DE works fine even after yesterdays update (win 7 everywere)

[Ronski]

Just like this :-

http://www.mediafire.com/file/aoz3zppgbcvrbia/2019-11-13_13-49-35.rar/file

Yes just like that.

I use Ripbot as is, no modifications apart from what's in the settings, and then it's only very minor things I change from default. Ripbot still works fine when not using DE, so my encodes currently take a little longer.

I have two encoding servers on the machine which hosts Ripbot, surely if it was a firewall issue they would still work? I also turned off my AV which made no difference. Just tried using the other machine (W7) as the host and it still doesn't work, so doesn't seem to be a Windows 10 thing.

A few years back I wrote some free software, and it's user base was very limited (unlike Ripbot), but support was hard and trying to keep up with every ones requests was time consuming. I did implement an auto update system, but the user had total control, there was also an option to backup the old version, and always a change log, there was also a beta version and release version

[guest]

Yes just like that.

I use Ripbot as is, no modifications apart from what's in the settings, and then it's only very minor things I change from default. Ripbot still works fine when not using DE, so my encodes currently take a little longer.

I have two encoding servers on the machine which hosts Ripbot, surely if it was a firewall issue they would still work? I also turned off my AV which made no difference. Just tried using the other machine (W7) as the host and it still doesn't work, so doesn't seem to be a Windows 10 thing.

A few years back I wrote some free software, and it's user base was very limited (unlike Ripbot), but support was hard and trying to keep up with every ones requests was time consuming. I did implement an auto update system, but the user had total control, there was also an option to backup the old version, and always a change log, there was also a beta version and release version

Yes, it's certainly strange how it's working for some, and not for others, and I guess that will be hard to diagnose, and fix.

And I have certainly been rubbing some people up the wrong way, but I'm just trying to get a point across....

[guest]

So I thought I'd post another "special" build of RB, which is v 1.25.0, core v 26-12-19, EC 1.17.0

It's the latest build, but has a couple of "handy" extra's

https://www.mediafire.com/file/96g4oa4mspostcq/Special_RipBot264.7z/file

I hope this will be useful for some....

I've had 24 downloads of this previous build I made available here:-

http://www.mediafire.com/file/vmd9rx70mmtdowi/RipBot264v1.25.0_%2528special%2529.rar/file

I think this was the build prior to the auto chunk combining (it worked for me)

I have removed these links, as they are really no longer relevant.

[Privateer5000]

Just double checked for the Firewall. I deactivated the Firewall and deleted the Rules concerning Ripbot but sadly no change. Running from another Directory did not change anything either. So i have to stick with 1.24 until this is fixed or perhaps Pauly is uploading his Version Core 22-10-19, EC 1.15.2. I do not know what else i could test for now.

Edit says: Pauly was faster :-) Thanks for the Upload

[Atak_Snajpera]

Let's investigate this issue STEP by STEP
1) Enable builtin system firewall and disable ALL 3rd party software like antivirus/antimalware
2) Open Firewall settings and remove all entries regarding encodingclient.exe and encodingserver.exe
https://i.postimg.cc/jtkFjZ22/Capture.png
3) Restart PC
4) Now when you click on EncodingServer.exe you should see this pop-up window
https://i.postimg.cc/02Ln0XKC/Capture2.png
5) Now run nmap.exe with this command line nmap.exe -Pn -p 1000 [YOUR IP HERE]
https://i.postimg.cc/v823cZQk/Capture6.png
Port is Open and EncodingServer also detected connection from nmap.

And here working connection between EncodingClient.exe and EncodingServer.exe

https://i.postimg.cc/nLx16skj/Capture5.png

[Privateer5000]

So, did Step 1-4.
This is Result for Step 5

https://i.ibb.co/WymysQR/Ripbot2.jpg (https://ibb.co/ncxcRTK)

It says Port 1000 is open, but the Encoding Server shows no reaction

[Atak_Snajpera]

Try this
nmap.exe -Pn -sT -p 1000 192.168.0.101

[Privateer5000]

This Command Line work and Server reacts:
https://i.ibb.co/LNZwVjv/Ripbot3.jpg (https://ibb.co/g4R1kq6)

But when i open the EncodingClient there is no reaction.

https://i.ibb.co/0jNmrH4/Ripbot4.jpg (https://ibb.co/7JcjKsM)

[Ronski]

Try this
nmap.exe -Pn -sT -p 1000 192.168.0.101

I'm getting the same results as Privateer, the above line caused the encoding server to react - this works for each encoding server when the port and IP address is changed as required, the previous one did not.

No reaction from the encoding client though.

[Atak_Snajpera]

I'm getting the same results as Privateer, the above line caused the encoding server to react, the previous one did not. No reaction from the encoding client.

That's still something.
This at least means that nothing is blocking connection and EncodingServer v1.16 is also ok. So it has to be some weird bug in EncodingClient...

[byteshare]

So it has to be some weird bug in EncodingClient...
Did you maybe put in a too broad of a variable for when you setup the block Pauly Dunne pc command ;)

[Atak_Snajpera]

Did you maybe put in a too broad of a variable for when you setup the block Pauly Dunne pc command ;)

Looks like my detection code is buggy... ;)
Ok. Jokes aside! I will post debug version of encodingclient for those reporting this issue. It should help me to track down where in code bad things happens...

[guest]

Did you maybe put in a too broad of a variable for when you setup the block Pauly Dunne pc command ;)

WOW, I saw what you did there.....

OMG, you're now my new BFF :)

[guest]

Looks like my detection code is buggy... ;)
Ok. Jokes aside! I will post debug version of encodingclient for those reporting this issue. It should help me to track down where in code bad things happens...

Not sure if you had a look at the screenshots I posted back at #17756, but I kinda thought back then, (even tho I was swapping and changing EC's & ES's around), that EC seemed to be looking for, and/or detecting the wrong ports.

[howzz]

i thought i'd report back to everyone that looks like another update came out over the weekend that updated Lmash and Core. looks like this latest update fixed the slow "gathering information" problem once again, and when you click on "start" encoding, it doesn't take forever for CPU cores to load up anymore.

i am going to image this version of RB and set it aside for backup.

but i agree with Paul Dunne in regards to version control, it would be a good idea to either post all the versions for users to download, or at the very least, have some kind of pop up that notifies user a new update is available and only upon clicking "yes", the update goes through. this way users can at least get a chance to backup their RB version prior to updating. otherwise we'll constantly run into downloading and updating to the broken version. obviously, everyone's setup is different. so what's broken for one person may very well work fine for others. but some kind of measure should be implemented regarding updates. we love updates and new features, and thank the author for that, but a layer of version control/protection is much needed.

[MCFish]

Atak, i see encodingclient/server uses ports 61200 and upwards (maybe) internally when it works (before the 1.16 update) maybe something with those ports...or the using of them.
https://i.postimg.cc/jjc8Vb16/rbcpold.jpg

[0gg]

Atak_Snajpera : thank you agin for this great job.

Could you be pleased to add these feature :

able to decode mxf files
adding custom timecode on mp4 or the timecode of the source

??

... to encode files from DSLR / Camera to lower resolution / proxy file but with the same timecode of the sources

0gg / augur.

[Atak_Snajpera]

Run this debug version and show me what messages you get
https://www.mediafire.com/file/9zic1q57t0gqqpt/client.exe/file

example screenshot from my machine
https://i.postimg.cc/1RZv7jXf/Capturex.png

[MCFish]

CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false

etc etc etc

[Atak_Snajpera]

CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false

etc etc etc

What happens if you enter 192.168.1.2:1000

[MCFish]

I assumed you meant i should enter it in 'Distributed Encoding Server IP'

it went so fast past me, but got something

CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
IP="192.168.1.2:1000"
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
CheckIP(0.0.0.0:0)=false
Starting workerthread
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
Connecting to "192.168.1.2:1000"
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false
CheckIP(192.168.0.8:1000)=false
CheckIP(192.168.0.2:1000)=false
CheckIP(192.168.0.12:1000)=false

There was a timout further down, guess it was for 192.168.1.2

[Privateer5000]

https://i.ibb.co/87HYRFd/Ripbot5.jpg (https://ibb.co/S3zf1qd)

With correct IP it does not find it, with 192.168.1.101:1000 it seems to find something but Connection timed out

[Atak_Snajpera]

I have located a bug in my function CheckIP. This function basically checks if ip values entered by user are in valid range of 0-255. It also checks other things like number of dots and if port is divisible by 1000 but that was not a problem here. Problem was that if you had 0 in ip (for example 192.168.0.1) then function was returning FALSE and hence no connection to server. This explains why it worked for me since I have IPs without zero value (192.168.1.100)

[Atak_Snajpera]

This version should work now
https://www.mediafire.com/file/zat5jd4as3wa1jl/EncodingClient.exe/file

[MCFish]

YES, great man. Thx
https://i.postimg.cc/sXsPCYgG/rbworks.jpg

[Atak_Snajpera]

Ok then BRACE YOURSELF! AUTO-UPDATE IS COMING!

[MCFish]

ill donate again ...;)

[byteshare]

Ok then BRACE YOURSELF! AUTO-UPDATE IS COMING!
Holding on to my butt ;)
Atak_Snajpera : thank you agin for this great job.

Could you be pleased to add these feature :

able to decode mxf files
adding custom timecode on mp4 or the timecode of the source

??

... to encode files from DSLR / Camera to lower resolution / proxy file but with the same timecode of the sources

0gg / augur.
I've been using a batch export time code (source) and import time code (encoded file); for when I need time codes. I usually don't need them if I'm not doing anything with QTGMC (de-interlace tool)
My camera uses MP4 and .ts files so I haven't used mxf files before, but I haven't needed to use time codes for those files (yet).

[howzz]

good job. see. put everyone's heads together and we can solve anything.

i'll donate some money to the cause come this fri pay day.

[Ronski]

Thanks Atak, working for me again as well - I had a zero in my IP address. I also sent a donation yesterday, thanks once again for all the hard work.

[guest]

Ok then BRACE YOURSELF! AUTO-UPDATE IS COMING!

Let's give it up for Atak....he's a Bloody Legend !!!

Well done, as we knew it would be, when you say "I give up" we don't really believe it. :)

It was a community effort of some of the RB users out there.

I'd just like to say that some things were said, that should have been a lot more tactful, but some of us express themselves differently to others.

I'll take note of the critisicm against me, and the comments I made, however I was also surprised with the support of some, for some suggestions I made, so it's not all bad.

We're all here to help with the ongoing improvement of this awesome free app, and I know that I won't stop using it, I have spent a lot of time & money setting up a range of systems so as to get the best benefit & performance out of RB, that I can, so I guess I was frustrated by the sudden unexpected DE problems, as DE, IS the awesome function of RB.

So hopefully we can move forward.....

So again Atak, thank you very much..........

PS:- I just wanted to add, that I just tested RB, (pre this latest fix), by changing my LAN's IP address from 192.168.0.5, to 192.168.1.5, and it worked, but now it's been fixed, that isn't an issue anymore. :)

[Dhry]

Outstanding! Thank you for resolving this issue Atak. Looking forward to the autoupdate.

Dhry

[t3nzin]

Thanks for the update Atak! Also, not sure if anyone else has reported it yet but RipBot is still prompting for the pinterf branch rather than the official Avisynth+ repo that merged his changes and is now on 3.4.0. Not a big thing just a small quality of life change for a future update :)

[Atak_Snajpera]

Auto-update core 2019.11.19

Added: support for mxf files (Material Exchange Format)

[mcjordan]

What about AviSynth v3.4?
https://github.com/AviSynth/AviSynthPlus/

When I starting RipBot I've a message like this:
AviSynth+ MT x64 not installed!

[Ryushin]

I'm not sure if it's just me or not. I've tried to trouble shoot this and I have not found anything that changes it's behavior.

I had to move back to spinning platters (six SAS 10K 2.5" drives, in a ZFS raidz2 with a 10GB SSD SLOG Write Cache) as the SSDs I had would not trim running behind LSI HBA. I'm waiting for Western Digital's new SA500 NAS SSDs to become available to purchase and I'll try that again.

Essentially it's taking 20-30 minutes of Starting.... before a chunk begins to encode, for each chunk. So after it finishes a chunk, another 20-30 minutes has to go by for it to do the next one. I have three encoding servers running.

There seems to be a lot of disk access on the drive containing the ripbot264temp folder.
https://i.postimg.cc/DJGCGtpX/Rip-Bot264-Disk-Access.png (https://postimg.cc/DJGCGtpX)

I'm encoding 4K rips from discs.

I've tried installing a fresh version of RipBot264 and let it perform all it's updates and there does not seem to be any difference. I've tried pulling in fresh jobs and there is no difference.

I've run disk benchmarks of the VM running RipBot264 and for spinning platters, it's not too bad and this is while the system was a little busy:
https://i.postimg.cc/LYr8CZVW/Rip-Bot264-Disk-MBs.png (https://postimg.cc/LYr8CZVW)
https://i.postimg.cc/hzJG10Vx/Rip-Bot264-Disk-IOPS.png (https://postimg.cc/hzJG10Vx)

I've added a 8 core 16 thread E5-2650 V2 chip, so I shoudn't be CPU bound anymore.

Is there anything else I can provide? I'm at a loss right now.

[Atak_Snajpera]

1) Abort your encoding
2) Double click on IndexStreams_RunAll.cmd for each job causing this problem
3) Restart encoding

It is a side effect of (good) changes in lsmash index structure. (2x size reduction). What you see is a reindexing process caused by incorrect (old) index file version (15 vs 16) stored in job folder.

[guest]

Updated Core (20-11-19), & ES (v1.16.1), Avisynth 3.4 & ffmpeg.

:thanks:

[Ryushin]

Couple of things.

Just had a machine update and when an encoding server starts it complains about "AviSynth+ MT x64 not installed!". Running as administrator, I ran Tools\Avisynth\install.cmd and it does not seem to make a difference. I tried Tools\Avisynth\Avisynth+\uninstall.cmd and installed again and that did not make a difference, it still complains. I'll dig into this a bit later.


I tried the IndexStreams_RunAll.cmd and it did not make any difference. Though, I did discover something interesting. When the very long slow "Starting...." of chunks, or the 10-20 minutes of "Gathering Information" to pick a second audio stream or leaving the Avisynth Menu. It only occurs if RipBot264 demuxes from a ripped 4K disc. If I use MakeMKV to make a mkv of the 4K source, and then pull that into RipBot264, everything behaves normally and takes seconds to complete. Only extra bit is for the EncodingClient to copy the large mkv before starting. So at least that is a workaround for now.

[Atak_Snajpera]

Couple of things.

Just had a machine update and when an encoding server starts it complains about "AviSynth+ MT x64 not installed!". Running as administrator, I ran Tools\Avisynth\install.cmd and it does not seem to make a difference. I tried Tools\Avisynth\Avisynth+\uninstall.cmd and installed again and that did not make a difference, it still complains. I'll dig into this a bit later.


I tried the IndexStreams_RunAll.cmd and it did not make any difference. Though, I did discover something interesting. When the very long slow "Starting...." of chunks, or the 10-20 minutes of "Gathering Information" to pick a second audio stream or leaving the Avisynth Menu. It only occurs if RipBot264 demuxes from a ripped 4K disc. If I use MakeMKV to make a mkv of the 4K source, and then pull that into RipBot264, everything behaves normally and takes seconds to complete. Only extra bit is for the EncodingClient to copy the large mkv before starting. So at least that is a workaround for now.
Latest version of EncodingServer.exe is v1.16.1

[Ryushin]

Just had a machine update and when an encoding server starts it complains about "AviSynth+ MT x64 not installed!". Running as administrator, I ran Tools\Avisynth\install.cmd and it does not seem to make a difference. I tried Tools\Avisynth\Avisynth+\uninstall.cmd and installed again and that did not make a difference, it still complains. I'll dig into this a bit later.


Uninstalled Avisynth software package that I had installed. Copied the RipBot264 from the working machine over to the problem one and everything came up fine.

[userx]

Hello
I've a mkv-file which has 1 video, 2 audio and 37 subtitle tracks. After demuxing i get an Error:
FFVideoSource: No video track found
(D:\Temp\RipBot264temp\job2\getinfo.avs, line 2)

I don't want to keep all the subtitles. Is there a workaround or a fix for this issue?

[guest]

Hello
I've a mkv-file which has 1 video, 2 audio and 37 subtitle tracks. After demuxing i get an Error:
FFVideoSource: No video track found
(D:\Temp\RipBot264temp\job2\getinfo.avs, line 2)

I don't want to keep all the subtitles. Is there a workaround or a fix for this issue?

You could give MKVToolNix a try....

[userx]

You could give MKVToolNix a try....

Thanks for the hint but maybe to prevent future requests the track limit could be raised.

[Atak_Snajpera]

Forget about buggy ffvideosource. Use lsmash.

[userx]

Use lsmash.
I'm using FFMS2 due to i run into issues if use LSMASH.
Encoding client runs into error:
....video.mkv.ffindex_track00.kf.txt does not exist. Autoclosing

[stax76]

Forget about buggy ffvideosource. Use lsmash.

What's the biggest issue right now? FFVideoSource is still the default in staxrip.

[Atak_Snajpera]

What's the biggest issue right now? FFVideoSource is still the default in staxrip.

Ffvideosource is not frame accurate for example

https://i.imgsafe.org/a3/a32816de27.png

[stax76]

What's the most common scenario where frame accuracy is needed?

[slalom]

Hello
I've a mkv-file which has 1 video, 2 audio and 37 subtitle tracks. After demuxing i get an Error:
FFVideoSource: No video track found
(D:\Temp\RipBot264temp\job2\getinfo.avs, line 2)

I don't want to keep all the subtitles. Is there a workaround or a fix for this issue?
I get that message all the time
Use MKV Merge and reduce subtitles <30

[Atak_Snajpera]

What's the most common scenario where frame accuracy is needed?

When you use a lot of Trim function. for example Trim(100,200)+Trim(300,400)+Trim(600,700)

[Ryushin]

I tried the IndexStreams_RunAll.cmd and it did not make any difference. Though, I did discover something interesting. When the very long slow "Starting...." of chunks, or the 10-20 minutes of "Gathering Information" to pick a second audio stream or leaving the Avisynth Menu. It only occurs if RipBot264 demuxes from a ripped 4K disc. If I use MakeMKV to make a mkv of the 4K source, and then pull that into RipBot264, everything behaves normally and takes seconds to complete. Only extra bit is for the EncodingClient to copy the large mkv before starting. So at least that is a workaround for now.

Atak, have you had time to look at this and see if it is a bug or something unique to me? If it is a bug and you're working on it, I'll leave my 15 or so 4K jobs there to hopefully have a fix soon. If your cannot confirm it or don't have time to look at it, let me know and I'll start converting those disc rips to mkv files, delete and re-add them to RB. Thanks much.

[Atak_Snajpera]

I do not have access to 4k BD at all...
However I would like to compare lsmash indexes (.lwi files) for mkv created by makemkv and ripbot.

[Ryushin]

I do not have access to 4k BD at all...
However I would like to compare lsmash indexes (.lwi files) for mkv created by makemkv and ripbot.

Ask and ye shall receive:
https://cloud.chrisdos.com/index.php/s/2BykG9ER9RrxM2E

BTW, detecting boarders took seconds from the MakeMKV 4K compared to 10-20 minutes from the demuxed version.

[Atak_Snajpera]

What happens if you take that video.mkv from job folder and mux with mkvtoolnix to new mkv? Does it also cause the same problem?

[BLKMGK]

Curious about SeekTester, is this why you index the files and pull them apart prior to compression and then remux them?

I have an alternate cluster encoding program I've been working on for Linux with a friend and we don't pull files apart or reindex them as you do but it passes your test program so I'm curious. This program is a ways from being mature enough to release and cannot presently use filters the way RipBot does but it does build pieces of files across many computers and then combine them so it's not completely different. The added steps have always been confusing to me for this reason:confused:

[Ryushin]

What happens if you take that video.mkv from job folder and mux with mkvtoolnix to new mkv? Does it also cause the same problem?

I muxed the RB extracted video.mkv along with all the contents except the video from the Casino Royal MakeMKV mkv and made a new mkv which I imported into RipBot. Everything was still speedy with the new mkv.

Did the index files look the same?

[byteshare]

Ffvideosource is not frame accurate...
Something just clicked when you said this and showed the image with errors. When you say frame accurate you don't mean timing, but rather frames completely just get skipped when they are not "accurate"?
I haven't played with GPU decoding much but if that is more accurate than LSmash I'd even use that on the machines I can use it on.

[userx]

I get that message all the time
Use MKV Merge and reduce subtitles <30

I solved it by this way.
But if the recomendation is to use lsmash instead of FFMS2 there is still an issue corresponding with ...video.mkv.ffindex_track00.kf.txt which seems not to get created.

[Atak_Snajpera]

Are you sure you have latest version? Other people do not report this issue.
Btw. I'm using lsmash all the time. Ffms2 is dead for me.

[Atak_Snajpera]

Curious about SeekTester, is this why you index the files and pull them apart prior to compression and then remux them?

I have an alternate cluster encoding program I've been working on for Linux with a friend and we don't pull files apart or reindex them as you do but it passes your test program so I'm curious. This program is a ways from being mature enough to release and cannot presently use filters the way RipBot does but it does build pieces of files across many computers and then combine them so it's not completely different. The added steps have always been confusing to me for this reason:confused:

Seektester Has nothing to do with de mode. Frame accurate seeking is crucial in De mode so i wrote that tool to establish which source filter is the most reliable. Lsmash is super stable while ffms2 is not.

[Atak_Snajpera]

I muxed the RB extracted video.mkv along with all the contents except the video from the Casino Royal MakeMKV mkv and made a new mkv which I imported into RipBot. Everything was still speedy with the new mkv.

Did the index files look the same?

No they are different because video.mkv is just a raw .265. eac3to can only demux hevc streams. For some unknown reason lsmash has problem with raw hevc. Could you upload first 5GB of video.mkv. Use dgsplit. Do not use mkvtoolnix. I would like to see If i also get that huge delay.

[Ryushin]

No they are different because video.mkv is just a raw .265. eac3to can only demux hevc streams. For some unknown reason lsmash has problem with raw hevc. Could you upload first 5GB of video.mkv. Use dgsplit. Do not use mkvtoolnix. I would like to see If i also get that huge delay.

Link to file: https://cloud.chrisdos.com/index.php/s/fG7fHFoGPt5YTQN

Though, there might be at least one other thing going on. When I have a 4K source, I select two audio tracks. Most of the time the first track is a AC3 core from a TrueHD track, and the second track is a demux of the same TrueHD track. After the job has been created and I click to add the second track, the Gathering Information takes about 10-20 minutes. The MakeMKV job takes about 30-60 seconds.

[Atak_Snajpera]

Ok. So the problem may not be related with HEVC stream but with TRUE-HD. Interesting...
BTW. can you check if .lwi file for true-hd is already created in job folder before you select second audio (True-hd)

[Ryushin]

Ok. So the problem may not be related with HEVC stream but with TRUE-HD. Interesting...
BTW. can you check if .lwi file for true-hd is already created in job folder before you select second audio (True-hd)

I made a new job and it did make a .lwi file for the TrueHD stream. Is it normal that the .lwi for TrueHD will be 400MB in size? The TrueHD stream itself is 2100MB. The video .lwi file is only 14MB in size.

While pulling in the job, and it is saying "Gathering Information..." for 10-20 minutes, ffprobe is running during this time. Then "Extracting Frame..." takes another 5-10 minutes.

[Atak_Snajpera]

I made a new job and it did make a .lwi file for the TrueHD stream. Is it normal that the .lwi for TrueHD will be 400MB in size? The TrueHD stream itself is 2100MB. The video .lwi file is only 14MB in size.

While pulling in the job, and it is saying "Gathering Information..." for 10-20 minutes, ffprobe is running during this time. Then "Extracting Frame..." takes another 5-10 minutes.

Yes it is "normal". It was even worse in previous version of lsmash
https://forum.doom9.org/showthread.php?p=1889725#post1889725

Can you send me whole True-hd stream? The longest sample I currently have is only 10min long?

[Ryushin]

Yes it is "normal". It was even worse in previous version of lsmash
https://forum.doom9.org/showthread.php?p=1889725#post1889725

Can you send me whole True-hd stream? The longest sample I currently have is only 10min long?

Here is a full stream (from a different movie though as it was smaller):
https://cloud.chrisdos.com/index.php/s/YPB3HpNitCoxL3r

[guest]

Link to file: https://cloud.chrisdos.com/index.php/s/fG7fHFoGPt5YTQN

Though, there might be at least one other thing going on. When I have a 4K source, I select two audio tracks. Most of the time the first track is a AC3 core from a TrueHD track, and the second track is a demux of the same TrueHD track. After the job has been created and I click to add the second track, the Gathering Information takes about 10-20 minutes. The MakeMKV job takes about 30-60 seconds.

I'd just like to chime in here, and add my 2 cents worth..

I have a LOT of 4K mkv's with TrueHD audio, and I run them thru MkvToolNix (sometimes tsmuxer) to rip out everything except the video, and then if need be, I will run that thru RB.

I then extract & convert the THD, and process any subtitles.

Once that's done, it all goes back together with MkvToolNix.

[Atak_Snajpera]

Are you looking for ultimate encoding machine? ;)
https://i.postimg.cc/N0bzsVWr/Untitled-1.png

[Ronski]

Are you looking for ultimate encoding machine? ;)

Impressive results, but wouldn't that be the 3990X with 64 cores 128 threads, although not yet available.

PS. Mine was 21.2 i7-5820K @4.5Ghz, although your benchmark reports the stock speed.

[BLKMGK]

Missed out on a 3950 to add to my 3900 today :( That clustered with my couple of other machines would be sick - I'll find one eventually! :) I cannot imagine a Threadripper but the price is just a hair too much methinks!

Edit: My 3900X@4.1GHz, optimizing not really complete as I'm still figuring out AMD, came in at 56.7FPS FWIW

[LigH]

@Ronski: Then better run several jobs in parallel, instead of wasting so many threads on only one; e.g. 8 jobs each 8 cores, that would be impressive. And possibly hard to keep cool...

[guest]

@Ronski: Then better run several jobs in parallel, instead of wasting so many threads on only one; e.g. 8 jobs each 8 cores, that would be impressive. And possibly hard to keep cool...

I've been waiting to see if someone else comments on this...

You can't be talking about RB running parallel jobs, as you can't have more that once instance of RB going at any one time, on the same PC.

Also, RB only does 1 job at a time.

Am I missing some well hidden secret setting !!! (or is it something you can do this with VM's)

[tkwou]

Hi I just updated Ripbot264. I haven't used it for a couple weeks. The Distributed Encoding wasn't working, only on the Host computer. I then updated all the other computers but I still couldn't get the Distributed Encoding to work. It just kept connecting and disconnecting with all the other computers. Finally I checked each computer and realized that Encoding Server never updated on any of the other computers. They all remained on 1.14.0.0. I finally manually updated each one to 1.16.0.0 from the Host computer and everything is up and running again. Not sure what went wrong on all the other computers.

[guest]

Hi I just updated Ripbot264. I haven't used it for a couple weeks. The Distributed Encoding wasn't working, only on the Host computer. I then updated all the other computers but I still couldn't get the Distributed Encoding to work. It just kept connecting and disconnecting with all the other computers. Finally I checked each computer and realized that Encoding Server never updated on any of the other computers. They all remained on 1.14.0.0. I finally manually updated each one to 1.16.0.0 from the Host computer and everything is up and running again. Not sure what went wrong on all the other computers.

Yes, DE went thru a bit of a bad patch a week or so ago...

But you still don't seem to be completely up to date.., Server is 1.16.1.0

[tkwou]

Thanks, something is also broken in 2-pass in v1.25.0. The "job1_EncodeVideoPass2.cmd" is blank. The process stops after the first pass.

[guest]

So I did another "big ass" 4K encode today, first one in a while due to the DE problems....

It was a 3h:48m movie, originally approx 33 Gb, @ 14mb/s bitrate, I used crf 10, no filters, it ended up being approx 77 Gb @ 42mb/s.

RB reported the job to have taken 5h:33m to complete, but in reality it took 6h:45m, as there seems to be a little bit of a discrepancy with that ETA calculation.

All in all, I had a pretty good run, a couple of servers dropped out during the encode, and strangely enough, ES didn't show that they had done so.

So it was a good test for the newest build.

But there is something that I sincerely hope can be considered in the very near future, and that is some more servers, say from 16, up to 20, as I have a few more pc's that could help the cause.

I know I'm not on your Christmas Card list, Atak, but please consider, that's all I can ask.

[guest]

Thanks, something is also broken in 2-pass in v1.25.0. The "job1_EncodeVideoPass2.cmd" is blank. The process stops after the first pass.

Have you got the latest update yet ??

Don't do any 2 pass, so I can't help there.

[tkwou]

Is there a link to download the latest version? I've only been relying on the update program and so far nothing else has updated. I reverted back to v1.24.0 everything seems to work with 2-pass again.

Latest entries in update_log.txt
2019-11-28 10:22:18 : [ERROR] C:\Program Files (x86)\Ripbot264\updates\core.zip is corrupted. CRC32 mismatch. Expected value was A8535996 but got D41A7FD1
2019-11-28 10:22:18 : File C:\Program Files (x86)\Ripbot264\updates\core.zip has been deleted.
2019-11-28 10:22:18 : [ERROR] C:\Program Files (x86)\Ripbot264\updates\ffmpeg.zip is corrupted. CRC32 mismatch. Expected value was 1026A3C but got 9ACD5420
2019-11-28 10:22:18 : File C:\Program Files (x86)\Ripbot264\updates\ffmpeg.zip has been deleted.
2019-11-28 10:22:18 : Installation complete.
2019-11-28 10:22:20 : Next check after 2019-11-29 10:20:56

[tkwou]

Ok, well I decided to try editing the update.ini file so the CRC32 values would match and this managed to force an update. Now I'm at v1.25.0, the Client is at 1.15.1.0 and the Server is at 1.16.1.0. FFMPEG also updated too. I'll try another encode and see what happens.

[guest]

Ok, well I decided to try editing the update.ini file so the CRC32 values would match and this managed to force an update. Now I'm at v1.25.0, the Client is at 1.15.1.0 and the Server is at 1.16.1.0. FFMPEG also updated too. I'll try another encode and see what happens.

Hi tkwou,

Props to you for sorting that mess out.

Obviously some conflict between times & file names, etc.

Sadly, there aren't many options to get the "latest" version, as for what ever reason, and after many requests, you can ONLY get v1.24.0 from the first page.

So what you could try, is getting v1.24.0, and doing a large auto update which should bring you right up to date.

OR

I posted a pretty recent build that was the last one that worked before the DE problems...you could grab that, and then do an auto update on that...end result will be the same, just a lot less files in the update download.

So if you want to try mine, the link is on post #17777

https://forum.doom9.org/showthread.php?p=1890571#post1890571

[tkwou]

Thank you, so everything is working fine now that I have all the core files updated. DE is also working fine now that all the other computers are also updated. The 2 pass problem I was having has also been fixed by this update.

[guest]

Thank you, so everything is working fine now that I have all the core files updated. DE is also working fine now that all the other computers are also updated. The 2 pass problem I was having has also been fixed by this update.

:thanks: & :cool:

[Atak_Snajpera]

It was a 3h:48m movie, originally approx 33 Gb, @ 14mb/s bitrate, I used crf 10, no filters, it ended up being approx 77 Gb @ 42mb/s.
I still can't figure out why you keep increasing file size instead of decreasing it like rest of 99,999% people using ripbot264.

[ReinerSchweinlin]

It was a 3h:48m movie, originally approx 33 Gb, @ 14mb/s bitrate, I used crf 10, no filters, it ended up being approx 77 Gb @ 42mb/s.
CRF10? Why?

[guest]

I still can't figure out why you keep increasing file size instead of decreasing it like rest of 99,999% people using ripbot264.

Well, I guess you really don't need to figure it out...I'm happy to be that 0.001% that does things differently.

Like I state in my sig:- "Not poorly done, just doin' it my way !!!"

However, having said that, I can't recall anyone suggesting some "good" settings for retaining quality, AND small size, without the process taking forever !!!

---------------------------------------------------------------------------

So this was actually a bit of an upscaling "test", I chose the 1st of the "Lord Of The Rings" movies, the extended edition, which of course is a LONG 1080p x264 movie, and I wanted to see how it would turn out as a 2160p, x265.

So instead of using a 1 pass, 60,000kb/s run, I opted for CRF 10.

I've read on several forums that the optimal bitrate for 2160p x265, is between 45 - 65Mb/s, hence my settings.

And that was the result (size wise), quality....meh, couldn't really tell.

But one thing that didn't take very long was the combining & muxing at the end, thanks to the NVMe's (thanks for that).

So that's that !!! I just need to figure out what the problem is with a server dropping out, randomly.

So a question about the 2 "new" buttons in Encoding Client, that's pretty much only used between jobs (in a queue), correct ??

I have implemented another app, to shutdown a remote server, when it's not doing much.

And please keep in mind, the addition of more servers, in the not too distant future :)

[guest]

CRF10? Why?

Well, BECAUSE I CAN !!!!

I've noticed too many people are quick with critism (and I'm guilty of that, too), but not forth coming with suggestions (on certain subjects).

Please read my above reply to Atak, that might help.

What would you suggest for good quality, size & time taken, 4K encoding settings ??

[Atak_Snajpera]

Like I state in my sig:- "Not poorly done, just doin' it my way !!!"
Do not forget about FU sign as avatar under your nick...

[Ronski]

Well, BECAUSE I CAN !!!!

I've noticed too many people are quick with critism (and I'm guilty of that, too), but not forth coming with suggestions (on certain subjects).

Please read my above reply to Atak, that might help.

What would you suggest for good quality, size & time taken, 4K encoding settings ??

What screen size do what the resultant video on? I use H265 CRF22 and I can't tell the difference on a 120" screen, maybe if the two were side by side I could.

[guest]

What screen size do what the resultant video on? I use H265 CRF22 and I can't tell the difference on a 120" screen, maybe if the two were side by side I could.

Hey Ronski, I know you mean't to say "What screen size do you watch the resultant video on ?

I only have a 55" UHD screen, I would doubt that a 120" screen would have a good pixel size/resolution, it's still 3840 x 2160...I would suggest that it would be "cleaner" on a smaller screen.

Even viewing a 55" along side a 75/76" screen, the 55" looks "cleaner".

I will try your "setting's", so thanks for your info.

[guest]

Do not forget about FU sign as avatar under your nick...

Yes, the "old middle finger"...not necessarily FU, could be contempt or defiance..

I've often wondered what your avatar is all about...

[Atak_Snajpera]

I've often wondered what your avatar is all about...
Never heard of tekken game?

[guest]

Never heard of tekken game?

Not a gamer, so no, I wouldn't have a clue...thanks for the explanation :(

Hwoarang....

[Atak_Snajpera]

But one thing that didn't take very long was the combining & muxing at the end, thanks to the NVMe's (thanks for that).
This beta version of EncodingClient.exe is now combining chunks on fly as soon as they are encoded (yes code checks if order of completion is correct)
https://www.mediafire.com/file/zat5jd4as3wa1jl/EncodingClient.exe/file
If tests goes well I will push another auto-update.

[GZZ]

So I did another "big ass" 4K encode today, first one in a while due to the DE problems....

It was a 3h:48m movie, originally approx 33 Gb, @ 14mb/s bitrate, I used crf 10, no filters, it ended up being approx 77 Gb @ 42mb/s.


Dont know if you missed how crf works. It stands for constant rate factor and it tries is to keep at constant image quality without looking at the bitrate. So if your source is only 14Mbit, the crf 10 might oversize your output as it tries to use a higher bitrate. Having a higher bitrate then source wont make the picture better and its just wasting bits. What can help you on a 14Mbit source is to use x265 if source is x264, that will half the size of your input at the same quality and half the bitrate.

Using crf 18-20 is normal and gives a 1:1 quality at a lower birtrate. Also use default (medium) preset. Its a allround really good quality without insane encoding times. X265 is alot slower the x264 because the codec is more advanced and the compression is more difficult. use 2 pass if your going for a specific filesize. You never know the output filesize of crf.

[Ryushin]

What would you suggest for good quality, size & time taken, 4K encoding settings ??

I encode everything to CRF18 I want to keep for storage. I make a separate 1080p HDR from the 4K source at CRF21 for streaming purposes for my family and a 720p CRF21 from 1080p sources.

I've looked long and hard at 1080p blu-ray sources against the CRF18 x264 encodes. Only time I can tell any difference if I pause a single frame and look really close. When the video is running I cannot tell any difference from CRF18 and the original. From what I read x265 CRF is about 2 steps better than x264 in quality. So x265 CRF20 would look the same as x264 CRF18. Since switching to x265 for all my encodes, I'm still doing CRF18, so I'm getting better quality than if I used x264.

Try taking a good quality 4K and HD source and making a CRF18 x265 file and compare that to a CRF10 x265 file. I'm sure you will not be able to tell the difference.

I never upscale my sources. My Sony TV upscaling engine is excellent so there is no need.

I'm not a fan of grain as I feel it distracts from the film (and it also make the file size larger) so I use MDegrain2/3 with a setting usually from 100-600. Some films come out looking so much better it is startling. The Blade Runner 4K comes to mine. A grainy soupy mess and my wife and I did not like it. Once it went through MDegrain, holy smokes, it was like looking at a different film.

I try and shoot for a 2 hour HD source to be about 5GB in size and a 4K source to be about 15GB in size. I adjust MDegrain (if needed) settings until I reach something like that. For some super grainy sources, such as Blade Runner it still comes out to much larger and closer to 30GB. All depends on the sources.

Some computer 4K generated movies shrink so small that I go to CRF16 and they are still under 10GB.

[guest]

This beta version of EncodingClient.exe is now combining chunks on fly as soon as they are encoded (yes code checks if order of completion is correct)
https://www.mediafire.com/file/zat5jd4as3wa1jl/EncodingClient.exe/file
If tests goes well I will push another auto-update.

Awesome...

You really DO love this shit, don't you !!! :)

I will give this a run, at my next opportunity. (edit:- I tried this today, and I think I might have missed something, but it was probably the combining of chunks, der..... I only noticed the muxing part), why I said I missed something was because I was doing short encodes, and the combining time would have been short, anyway.

So can I suggest something ???

With the upcoming update, can you PLEASE, PLEASE, PLEASE consider a build version number revision (as a reference point to a substantial change), say at least v1.26.0.

And again, it would be really nice if you could also post a more recent build on page 1, as it's getting horribly close to 12 months old.

[guest]

Dont know if you missed how crf works. It stands for constant rate factor and it tries is to keep at constant image quality without looking at the bitrate. So if your source is only 14Mbit, the crf 10 might oversize your output as it tries to use a higher bitrate. Having a higher bitrate then source wont make the picture better and its just wasting bits. What can help you on a 14Mbit source is to use x265 if source is x264, that will half the size of your input at the same quality and half the bitrate.

Using crf 18-20 is normal and gives a 1:1 quality at a lower birtrate. Also use default (medium) preset. Its a allround really good quality without insane encoding times. X265 is alot slower the x264 because the codec is more advanced and the compression is more difficult. use 2 pass if your going for a specific filesize. You never know the output filesize of crf.

Hey GZZ, I think I have a bit of an idea, I went here :-

https://trac.ffmpeg.org/wiki/Encode/H.264

and here :-

https://trac.ffmpeg.org/wiki/Encode/H.265

and had a bit of a read...very interesting.

And yes, using crf, you have no idea what the final "size" will be, mind you, I don't really care what the final size is.

I have only tried 1 pass, as 2 pass would have to basically double the encoding time :(

[guest]

I encode everything to CRF18 I want to keep for storage. I make a separate 1080p HDR from the 4K source at CRF21 for streaming purposes for my family and a 720p CRF21 from 1080p sources.

I've looked long and hard at 1080p blu-ray sources against the CRF18 x264 encodes. Only time I can tell any difference if I pause a single frame and look really close. When the video is running I cannot tell any difference from CRF18 and the original. From what I read x265 CRF is about 2 steps better than x264 in quality. So x265 CRF20 would look the same as x264 CRF18. Since switching to x265 for all my encodes, I'm still doing CRF18, so I'm getting better quality than if I used x264.

Try taking a good quality 4K and HD source and making a CRF18 x265 file and compare that to a CRF10 x265 file. I'm sure you will not be able to tell the difference.

I never upscale my sources. My Sony TV upscaling engine is excellent so there is no need.

I'm not a fan of grain as I feel it distracts from the film (and it also make the file size larger) so I use MDegrain2/3 with a setting usually from 100-600. Some films come out looking so much better it is startling. The Blade Runner 4K comes to mine. A grainy soupy mess and my wife and I did not like it. Once it went through MDegrain, holy smokes, it was like looking at a different film.

I try and shoot for a 2 hour HD source to be about 5GB in size and a 4K source to be about 15GB in size. I adjust MDegrain (if needed) settings until I reach something like that. For some super grainy sources, such as Blade Runner it still comes out to much larger and closer to 30GB. All depends on the sources.

Some computer 4K generated movies shrink so small that I go to CRF16 and they are still under 10GB.

Hey Ryushin, :goodpost:

I am actually a little reluctant to use MDegrain on a large 4K source....wouldn't it take forever ???

I'd suggest that your crf 18 would be a good place to be, but I am still experimenting, before I settle on what I like. But yeah, I think around the 16 - 18 mark, will be it.

OR, I could just go to my high single pass settings, like I was going to in the first place, then you know approx what size you'll end up with.

I never upscale my sources. My Sony TV upscaling engine is excellent so there is no need.

My LG TV does a pretty good job, too...nearly EVERYTHING looks good on that :)

I'm not a fan of grain as I feel it distracts from the film (and it also make the file size larger) so I use MDegrain2/3 with a setting usually from 100-600.

I think the default for MDegrain, in RB is 400 (if I'm looking in the right place)

#Denoise
Loadplugin("D:\RipBot264v1.25.0\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
fv1=MAnalyse(super,isb=false,delta=1,overlap=4)
bv1=MAnalyse(super,isb=true,delta=1,overlap=4)
fv2=MAnalyse(super,isb=false,delta=2,overlap=4)
bv2=MAnalyse(super,isb=true,delta=2,overlap=4)
video=MDegrain2(video,super,bv1,fv1,bv2,fv2,thSAD=400)


I have to say, I haven't heard the phrase, "holy smokes" for such a long time...nice :)

[GZZ]

Hey Ryushin, :goodpost:

I am actually a little reluctant to use MDegrain on a large 4K source....wouldn't it take forever ???


MDegrain2 takes time on 4k source, but it also depends on the among og noise/film grain the source have. There is also a MDegrain1 and 3. Mdegrain1 is for light denoise, MDegrain2 is default and does a good job, if its still not enough, the MDegrain3. M3 will be slowest and M1 is faster then M2.

It could be nice to have a feature to select a single block (1-2 min) from the movie and run 3 small encodings with the above to see which one to use on the entire movie.

MDegrain 1 and 3 is not default in RB, you need to add them as a custom filter

[guest]

MDegrain2 takes time on 4k source, but it also depends on the among og noise/film grain the source have. There is also a MDegrain1 and 3. Mdegrain1 is for light denoise, MDegrain2 is default and does a good job, if its still not enough, the MDegrain3. M3 will be slowest and M1 is faster then M2.

It could be nice to have a feature to select a single block (1-2 min) from the movie and run 3 small encodings with the above to see which one to use on the entire movie.

MDegrain 1 and 3 is not default in RB, you need to add them as a custom filter

On the few movies that I have used M2 on, it seems to able to determine what it needs to do to that particular source :)

There is a little "tool" called "DGSplit", that can "cut" a user determined sized chunk out of a movie, and then you could do the test encodes on them.

Yes, I know that M2 is default, but I wouldn't have a clue of even where to start to add M1 or M3 as a custom filter (hint, hint)

[Ryushin]

I am actually a little reluctant to use MDegrain on a large 4K source....wouldn't it take forever ???

I'd suggest that your crf 18 would be a good place to be, but I am still experimenting, before I settle on what I like. But yeah, I think around the 16 - 18 mark, will be it.

I think the default for MDegrain, in RB is 400 (if I'm looking in the right place)

#Denoise
Loadplugin("D:\RipBot264v1.25.0\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
fv1=MAnalyse(super,isb=false,delta=1,overlap=4)
bv1=MAnalyse(super,isb=true,delta=1,overlap=4)
fv2=MAnalyse(super,isb=false,delta=2,overlap=4)
bv2=MAnalyse(super,isb=true,delta=2,overlap=4)
video=MDegrain2(video,super,bv1,fv1,bv2,fv2,thSAD=400)


I have to say, I haven't heard the phrase, "holy smokes" for such a long time...nice :)

I like older phrases. :)

MDegrain2 adds to the encoding time for sure. It usually doubles or triples the time to encode. My end goal though is for the movie to look just as good as the source or better than the source with the grain removed. So for me, encoding time isn't as important as the final quality.

I know there are some purists that want grain and want it to look like a film and to only run at 24fps (no soap opera effect). To me, I want to be wrapped up in the film, so i want it to be clean and I like that the TV interpolate frames to make it have smooth motion.

The default for RipBot264 MDegrain2 is 400 like you mentioned. I've been after Atak to extend the MDegrain option to choose 1-3 and thSAD input box with a range between 0-800.

My MDegrain3 looks like this:
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
bv2=MAnalyse(super,blksize=8,isb=true,delta=2,overlap=4)
bv3=MAnalyse(super,blksize=8,isb=true,delta=3,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
fv2=MAnalyse(super,blksize=8,isb=false,delta=2,overlap=4)
fv3=MAnalyse(super,blksize=8,isb=false,delta=3,overlap=4)
video=MDegrain3(video,super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)

I usually edit the avisynth script in the edit window to change the thSAD value. It's very rare that I need to go with MDegrain3, but when I do, I just replace Atak's line with mines.

Those links ffmpeg links you sent look good. It looks like I was mistaken then. FFMPEG says it's a 5 value difference from x264 to x265 to have it look the same. So x265 CRF18 should look the same as x264 CRF13. So for the CRF18 with x265 would be visually indistinguishable from the source.

[Atak_Snajpera]

This should explain why I decided to not use MDegrain3
https://i.postimg.cc/NFB44R4L/1.png https://i.postimg.cc/KznfB0td/2.png

Quick summary:
MDegrain1 vs No Denoise
Encoding speed reduction = 1.72x
File size reduction = 1.49x
Size/Speed ratio = 0.87

MDegrain2 vs No Denoise
Encoding speed reduction = 2.70x
File size reduction = 1.75x
Size/Speed ratio = 0.65

MDegrain3 vs No Denoise
Encoding speed reduction = 3.57x
File size reduction = 1.92x
Size/Speed ratio = 0.54

[Ryushin]

This should explain why I decided to not use MDegrain3
https://i.postimg.cc/NFB44R4L/1.png https://i.postimg.cc/KznfB0td/2.png

Quick summary:
MDegrain1 vs No Denoise
Encoding speed reduction = 1.72x
File size reduction = 1.49x
Size/Speed ratio = 0.87

MDegrain2 vs No Denoise
Encoding speed reduction = 2.70x
File size reduction = 1.75x
Size/Speed ratio = 0.65

MDegrain3 vs No Denoise
Encoding speed reduction = 3.57x
File size reduction = 1.92x
Size/Speed ratio = 0.54

On some very grainy 4K sources, MDegrain3 has cleaned up source enough to further reduce the size by 5-10GB. That is pretty significant. MDegrain3 also did wonders with the Walking Dead series.

[guest]

This should explain why I decided to not use MDegrain3

On some very grainy 4K sources, MDegrain3 has cleaned up source enough to further reduce the size by 5-10GB. That is pretty significant. MDegrain3 also did wonders with the Walking Dead series.

I'd just like to add my 2 cents worth, here...

It would be interesting to see that info for 4K x265 filtering.

Graphs, data, % etc, are all well and good, but the final test is one's eye's !!!

As 4K is becoming the norm, it needs to work for that.

I'm probably wrong, but I'm sure it wouldn't take very long for Atak to implement more MDegrain options, he's got M2 setup, all that possibly needs doing is edit the scripts for M1 & M3, add them to the drop down list, and there you have it....no more discussion.

Now who here, uses KNLMeansCL ???, it certainly doesn't seem to be mentioned very often, and I think it being fundamentally a GPU based filter, you need a powerful GPU, if I recall, you also need particular software/drivers to be able to use it, and it's SLOW....so combining M2 & KNL, would take days to encode a feature length movie, especially 4K.

There's 1 x KNL (on it's own), and 3 with M2, and 1 x M2 (on its own) option's, in the dropdown.

Personally I miss, HQDN3D, I use to use that a lot.

[Ryushin]

Graphs, data, % etc, are all well and good, but the final test is one's eye's !!!

As 4K is becoming the norm, it needs to work for that.

Now who here, uses KNLMeansCL ???, it certainly doesn't seem to be mentioned very often, and I think it being fundamentally a GPU based filter, you need a powerful GPU, if I recall, you also need particular software/drivers to be able to use it, and it's SLOW....so combining M2 & KNL, would take days to encode a feature length movie, especially 4K.

Personally I miss, HQDN3D, I use to use that a lot.

May I suggest buying the first Blade Runner in 4K and running MDegrain2-600 against that and then Watching the original. Especially the first scene when he gives the voidcomp (sp?) test in the hazy florescent room. MDegrain is so amazing in that it took out the soupy grain but left details like the cigarette smoke clear to see and make out. Maybe I'll cut out the two scenes and upload them.

I used to use HQDN3D. It was far faster than MDegrain2 and it was for a time, the only option available. The quality results though, vastly different between the two. So different I actually went back re-encoded a few sources with the new MDegrain and this was before Atak implemented threading and I was only getting 2fps. The quality was worth the wait. Now HQDN3D was also good to me when using in conjunction with MDegrain to remove a bit more harder to remove grain. But since HQDN3D cannot to 10Bit color (if I remember right) Atak removed it. That is when I switched to using MDegrain3.

[Atak_Snajpera]

May I suggest buying the first Blade Runner in 4K and running MDegrain2-600 against that and then Watching the original. Especially the first scene when he gives the voidcomp (sp?) test in the hazy florescent room. MDegrain is so amazing in that it took out the soupy grain but left details like the cigarette smoke clear to see and make out. Maybe I'll cut out the two scenes and upload them.

I used to use HQDN3D. It was far faster than MDegrain2 and it was for a time, the only option available. The quality results though, vastly different between the two. So different I actually went back re-encoded a few sources with the new MDegrain and this was before Atak implemented threading and I was only getting 2fps. The quality was worth the wait. Now HQDN3D was also good to me when using in conjunction with MDegrain to remove a bit more harder to remove grain. But since HQDN3D cannot to 10Bit color (if I remember right) Atak removed it. That is when I switched to using MDegrain3.

Yes upload IT please.

[slalom]

That is when I switched to using MDegrain3.
I don't see that in the menu, is that only for 4K?

[Atak_Snajpera]

I don't see that in the menu, is that only for 4K?

You have to create custom script


#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
bv2=MAnalyse(super,blksize=8,isb=true,delta=2,overlap=4)
bv3=MAnalyse(super,blksize=8,isb=true,delta=3,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
fv2=MAnalyse(super,blksize=8,isb=false,delta=2,overlap=4)
fv3=MAnalyse(super,blksize=8,isb=false,delta=3,overlap=4)
video=MDegrain3(video,super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)

[Ryushin]

I've cut 7:39 from the beginning of Blade Runner which includes the voight kampff test. The original and the Mdegrain3-800 is here. I'l leave this up for a week and then take it down.

https://cloud.chrisdos.com/index.php/s/YQJ2Y6LEdPcweEP

[stryker412]

Atak, I'm finalizing a new gaming/encoding build. With regards to Ripbot only, what do you think of this build? Any hardware not ripbot friendly? I'm looking at the 5700XT vs 2070 Super for GPU.

https://pcpartpicker.com/user/Stryker412/saved/#view=mW7zNG

[Atak_Snajpera]

Atak, I'm finalizing a new gaming/encoding build. With regards to Ripbot only, what do you think of this build? Any hardware not ripbot friendly? I'm looking at the 5700XT vs 2070 Super for GPU.

https://pcpartpicker.com/user/Stryker412/saved/FwQXvK

This part list is private.

[stryker412]

This part list is private.

Sorry, fixed the original link.

[Atak_Snajpera]

Sorry, fixed the original link.

Everything looks great except that SSD... QLC? Really? You won't be happy with 100 MiB/s real write speed...
http://thessdreviewcdn2.thessdreview1.netdna-cdn.com/wp-content/uploads/2018/07/Intel-SSD-660P-HDTune.png

[slalom]

You have to create custom script


#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
bv2=MAnalyse(super,blksize=8,isb=true,delta=2,overlap=4)
bv3=MAnalyse(super,blksize=8,isb=true,delta=3,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
fv2=MAnalyse(super,blksize=8,isb=false,delta=2,overlap=4)
fv3=MAnalyse(super,blksize=8,isb=false,delta=3,overlap=4)
video=MDegrain3(video,super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)
Any chance of adding this to the menu for novices like me?

[stryker412]

Everything looks great except that SSD... QLC? Really? You won't be happy with 100 MiB/s real write speed...
http://thessdreviewcdn2.thessdreview1.netdna-cdn.com/wp-content/uploads/2018/07/Intel-SSD-660P-HDTune.png

I currently have a 2nd drive for ripbot encodes. What drive type do you suggest as a 2nd drive?

[Atak_Snajpera]

Any chance of adding this to the menu for novices like me?

Just save that to a file in Tools\AviSynth plugins\Scripts\Custom

Just make sure path in LoadPlugin is correct

[Atak_Snajpera]

I currently have a 2nd drive for ripbot encodes. What drive type do you suggest as a 2nd drive?

Something decent using at least TLC... Just avoid QLC like plague because write endurance is also lower than in TLC.

[stryker412]

Something decent using at least TLC... Just avoid QLC like plague because write endurance is also lower than in TLC.

How about this one?
https://www.newegg.com/crucial-mx500-1tb/p/N82E16820156178?Item=N82E16820156178

[guest]

You have to create custom script


#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
bv2=MAnalyse(super,blksize=8,isb=true,delta=2,overlap=4)
bv3=MAnalyse(super,blksize=8,isb=true,delta=3,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
fv2=MAnalyse(super,blksize=8,isb=false,delta=2,overlap=4)
fv3=MAnalyse(super,blksize=8,isb=false,delta=3,overlap=4)
video=MDegrain3(video,super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)

This is Gold !!! :)

Any chance of a script for MDegrain1, to complete the set ? :rolleyes:

[Atak_Snajpera]

How about this one?
https://www.newegg.com/crucial-mx500-1tb/p/N82E16820156178?Item=N82E16820156178

That one is sataIII not NVMe!

[Atak_Snajpera]

This is Gold !!! :)

Any chance of a script for MDegrain1, to complete the set ? :rolleyes:

#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
video=MDegrain1(video,super,bv1,fv1,thSAD=400)

[stryker412]

That one is sataIII not NVMe!


Gotcha. Found this one but it's much more expensive.

https://www.amazon.com/Samsung-970-EVO-1TB-MZ-V7E1T0BW/dp/B07BN217QG/ref=sr_1_1?keywords=nvme&psr=EY17&qid=1575306879&s=cyber-monday&smid=ATVPDKIKX0DER&sr=1-1

[guest]

#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
video=MDegrain1(video,super,bv1,fv1,thSAD=400)

:thanks:

[stryker412]

Atak what about nvidia vs Radeon for GPU?

[guest]

Gotcha. Found this one but it's much more expensive.

https://www.amazon.com/Samsung-970-EVO-1TB-MZ-V7E1T0BW/dp/B07BN217QG/ref=sr_1_1?keywords=nvme&psr=EY17&qid=1575306879&s=cyber-monday&smid=ATVPDKIKX0DER&sr=1-1

Or

https://www.amazon.com/XPG-SX8200-Gen3x4-3000MB-ASX8200PNP-1TT-C/dp/B07K1J3C23/ref=mp_s_a_1_2?keywords=adata+nvme&qid=1575307192&sr=8-2

I have some 512Gb versions, but I think I would go for EVO Plus, next. Pro's would be nice, but too exy :(

[guest]

Atak what about nvidia vs Radeon for GPU?

Depends on what decoding and/or encoding filters you want to use...

[stryker412]

Or

https://www.amazon.com/XPG-SX8200-Gen3x4-3000MB-ASX8200PNP-1TT-C/dp/B07K1J3C23/ref=mp_s_a_1_2?keywords=adata+nvme&qid=1575307192&sr=8-2

What's the advantage for the AData? They're the same price and look to have similar speeds.

[guest]

What's the advantage for the AData? They're the same price and look to have similar speeds.

Probably nothing...just another option....

I would probably go for at least the EVO Plus...

[stryker412]

Depends on what decoding and/or encoding filters you want to use...

I generally just use Atak's presets. Starting to move towards all h265 but on my current machine (i7-950) it takes all day for one encode.

[slalom]

Just save that to a file in Tools\AviSynth plugins\Scripts\Custom
Added both 1 & 3

Just make sure path in LoadPlugin is correct
??
I found the new scripts in custom script instead of Denoise, is that OK?

Hello
I've a mkv-file which has 1 video, 2 audio and 37 subtitle tracks. After demuxing i get an Error:
FFVideoSource: No video track found
(D:\Temp\RipBot264temp\job2\getinfo.avs, line 2)

I don't want to keep all the subtitles. Is there a workaround or a fix for this issue?
Having >30 subtitle tracks on the source is still an issue. Keep them <30 before adding the job

[Ripmann]

Hi, long time RipBot user here. Really great program. Thank you for your unwavering dedication and generosity.

I have a question regaring the entries in the audio PROFILE dropdown list. Can these default entries be changed or new ones added? I constantly need to use either 576kbps or 512kbps for 5.1 and 384kbps for 2.0 AC3, so I'm always forced to look for the job's jobX_EncodeAudio1.cmd and edit the file in manually after each set up. It's pretty time consuming even for a single job, especially since the .cmd file resets every time you re-open the job in RipBot for any reason, and the program gives an error if you try to mark the file as read only. And the worst part is that if you're doing size-locked 2-pass encoding, you have to estimate the size difference of the audio file manually and manually deduct the extra megabytes from the total. All in all, the lack of these entries is a major inconvenience in my daily RipBot usage.

If modifying the profile entries via config or something isn't possible, could you please consider adding 5.1 512, 5.1 576, and 2.0 384 kbps to the default AC3 profiles list? Not sure about other users, but I really use these three A LOT.

----------------------------
P.S.: Since the forum enforces a mandatory 5-day posting block after registering, I came up with some additional things to say by the time I was able to post:

1) Why does Preview Script sometimes open several preview windows when clicked multiple times, and sometimes only one? I often would like to preview multiple denoise or crop settings, to quickly compare quality and so on. When I try clicking on Preview Script, sometimes I do get several job#_preview.avs MPC-HC pop-ups open, but usually I do not. Since the preview player has "Open a new player for each media file played" setting turned on, I'm assuming that RipBot checks if a preview window is already running and prevents opening another one if it does.

1b) Speaking of the preview popup, you may want to go to the settings of ...\Tools\mpc\mpc-hc64.exe, and set "Store settings in .ini file" (in Player->Other) to true. This way, it'll keep the default settings you want to use in the same folder as the exe without reading custom settings from actual installations of MPC-HC. I happen to use MPC-HC separately, and RipBot's preview exe had been using my custom settings (FFDshow filters, custom saturation and contrast, etc.) in its previews until I disabled it.

2) As another potential feature request, please consider separating the "Temp Files" path setting into something like "Jobs Temp Files" and "Encoding Temp Files". As an example why it could be very useful, I use a temp RAM-based drive that wipes with every restart. I would very much like to have my jobs files be kept on an SSD (since I want to keep them until I remove them manually), and temporary video.265 encoding files go to the RAM drive, since these files are much more write-intensive and they get auto-deleted after muxing to "Default Output Path" anyway. In short, RAM drives are perfect for temporary encoded tracks but not for jobs files. The ability to split the two into separate locations would help a lot.

That's pretty much it. I realize that you're probably flooded with requests, so regardless of your answer, once again, thank you very much.

[stryker412]

Or

https://www.amazon.com/XPG-SX8200-Gen3x4-3000MB-ASX8200PNP-1TT-C/dp/B07K1J3C23/ref=mp_s_a_1_2?keywords=adata+nvme&qid=1575307192&sr=8-2

I have some 512Gb versions, but I think I would go for EVO Plus, next. Pro's would be nice, but too exy :(

Crazy thing is, about 1.5hrs after you posted this it went to a lightning deal but I didn't know about it in time. I'm on the waitlist.

[Atak_Snajpera]

Hi, long time RipBot user here. Really great program. Thank you for your unwavering dedication and generosity.

I have a question regaring the entries in the audio PROFILE dropdown list. Can these default entries be changed or new ones added? I constantly need to use either 576kbps or 512kbps for 5.1 and 384kbps for 2.0 AC3, so I'm always forced to look for the job's jobX_EncodeAudio1.cmd and edit the file in manually after each set up. It's pretty time consuming even for a single job, especially since the .cmd file resets every time you re-open the job in RipBot for any reason, and the program gives an error if you try to mark the file as read only. And the worst part is that if you're doing size-locked 2-pass encoding, you have to estimate the size difference of the audio file manually and manually deduct the extra megabytes from the total. All in all, the lack of these entries is a major inconvenience in my daily RipBot usage.

If modifying the profile entries via config or something isn't possible, could you please consider adding 5.1 512, 5.1 576, and 2.0 384 kbps to the default AC3 profiles list? Not sure about other users, but I really use these three A LOT.

----------------------------
P.S.: Since the forum enforces a mandatory 5-day posting block after registering, I came up with some additional things to say by the time I was able to post:

1) Why does Preview Script sometimes open several preview windows when clicked multiple times, and sometimes only one? I often would like to preview multiple denoise or crop settings, to quickly compare quality and so on. When I try clicking on Preview Script, sometimes I do get several job#_preview.avs MPC-HC pop-ups open, but usually I do not. Since the preview player has "Open a new player for each media file played" setting turned on, I'm assuming that RipBot checks if a preview window is already running and prevents opening another one if it does.

1b) Speaking of the preview popup, you may want to go to the settings of ...\Tools\mpc\mpc-hc64.exe, and set "Store settings in .ini file" (in Player->Other) to true. This way, it'll keep the default settings you want to use in the same folder as the exe without reading custom settings from actual installations of MPC-HC. I happen to use MPC-HC separately, and RipBot's preview exe had been using my custom settings (FFDshow filters, custom saturation and contrast, etc.) in its previews until I disabled it.

2) As another potential feature request, please consider separating the "Temp Files" path setting into something like "Jobs Temp Files" and "Encoding Temp Files". As an example why it could be very useful, I use a temp RAM-based drive that wipes with every restart. I would very much like to have my jobs files be kept on an SSD (since I want to keep them until I remove them manually), and temporary video.265 encoding files go to the RAM drive, since these files are much more write-intensive and they get auto-deleted after muxing to "Default Output Path" anyway. In short, RAM drives are perfect for temporary encoded tracks but not for jobs files. The ability to split the two into separate locations would help a lot.

That's pretty much it. I realize that you're probably flooded with requests, so regardless of your answer, once again, thank you very much.

I'm curious why do you have to use those particular bitrates? Is this some kind of hardware limitation? All blu-ray movies just use 640kbps for 5.1 AC3 and Stereo is often encoded as 224-256kbps.

[guest]

I generally just use Atak's presets. Starting to move towards all h265 but on my current machine (i7-950) it takes all day for one encode.

Do you have more than one PC ???

Do you use Distributed Encoding ??? (you can use this one only 1 PC, can speed things up a little)

Unless you are using certain filters (KNLMeansCL) that require a powerful GPU, you really don't need anything too special, unless you're doing 4K, and have a 4K monitor.

[guest]

I've cut 7:39 from the beginning of Blade Runner which includes the voight kampff test. The original and the Mdegrain3-800 is here. I'l leave this up for a week and then take it down.

https://cloud.chrisdos.com/index.php/s/YQJ2Y6LEdPcweEP

Hey, I thought I'd grab these as well, to see what you'd done...

OMFG, that is SO much better, that would be worth a long encode, for that transformation...well done !!!

edit:- So like I said, I downloaded these to have a look for myself, and I also decided to "test" the (new to me) MDegrain scripts that Atak kindly provided, and I loaded the file 3 times, each with a different MDegrain script, and then when I went to encode, it started to process the chunks and the it errored, stating it couldn't find something in the job folder.....so I checked it Mediainfo, and was surprised to see the frame rate

Video
ID : 1
ID in the original source medium : 4113 (0x1011)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 7 min 24 s
Bit rate : 50.0 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 380.645 FPS
Original frame rate : 23.976 (24000/1001) FPS

So the only thing I could think of was to run it thru Handbrake to bring it down a touch (60fps), loaded it back into RB, away we go...


BTW, how long did it take to encode the MDegrain3 sample ??

[BLKMGK]

I'd just like to add my 2 cents worth, here...

It would be interesting to see that info for 4K x265 filtering.

Graphs, data, % etc, are all well and good, but the final test is one's eye's !!!

As 4K is becoming the norm, it needs to work for that.

I'm probably wrong, but I'm sure it wouldn't take very long for Atak to implement more MDegrain options, he's got M2 setup, all that possibly needs doing is edit the scripts for M1 & M3, add them to the drop down list, and there you have it....no more discussion.

Now who here, uses KNLMeansCL ???, it certainly doesn't seem to be mentioned very often, and I think it being fundamentally a GPU based filter, you need a powerful GPU, if I recall, you also need particular software/drivers to be able to use it, and it's SLOW....so combining M2 & KNL, would take days to encode a feature length movie, especially 4K.

There's 1 x KNL (on it's own), and 3 with M2, and 1 x M2 (on its own) option's, in the dropdown.

Personally I miss, HQDN3D, I use to use that a lot.

Actually, I use the KNLMeansCL filter with MDegrain2 quite a bit. I choose this for older stuff that shows noticeable grain or artifacts in the Preview Script. The cleanup it did on a BluRay version of the older Red Dawn movie was jaw dropping. I've tested it with both high end and fairly pedestrian video cards - the difference in speed was negligible so purchasing a high end card to use this seems pointless. I have mostly 1050 NVIDIA cards in my machines (one has a GTX 680) and the NVIDIA drivers automatically load the desired OpenCL driver. No muss no fuss, this is one VERY nice filter IMO. Currently not doing 4K video here as the HDR stuff is a hassle. I do use CRF 20 on say TV shows but movies are always CRF 18 and look terrific, X.265 Slow is my profile of choice and with a number of machines I make fairly short work of videos. When I can track down a Ryzen 3950X it should move along a good bit quicker! For movies with filtering I get just as bit over 30FPS so it's certainly not a day long affair when you've clustered hardware.

P.S. I noticed awhile back someone mentioned the scaling issue with cores. To take care of this run multiple copies of the encoding server - problem solved! You can even set core affinity I think but I haven't found the need.

[Ripmann]

I'm curious why do you have to use those particular bitrates? Is this some kind of hardware limitation? All blu-ray movies just use 640kbps for 5.1 AC3 and Stereo is often encoded as 224-256kbps.

Hi Atak, thanks for reading my post.

I use 576kbps for a specialized case, but speaking more broadly, it's a size-related limitation. Here are some common use scenarios that I think could apply to other users besides me:

With longer tracks, higher bitrate audio can take up too much space even when compressed. I very often use 512-576 as a compromise between being forced to go lower than 640 but without the somewhat noticeable quality loss of 448. For example, I have a 5GB DTSMA track in queue right now that encodes to 904MB at 640kbps, 814MB at 576kbps, 723MB at 512kbps, and 633MB at 448kbps, so jumping from 448 to 640 is almost 50% in size difference. Having an option to pick an intermediate step between these two is often very useful, especially when recordable media sizes are fixed but track lengths can vary greatly.

For stereo it's a bit different case. Here I usually use it in a situation where I prefer the best quality I can afford. So I want at least 320kbps, close to the highest level supported by the MP3 standard, but I often go with an overkill of 384kbps since I want as little quality loss as possible and the difference in size is relatively negligible. Again, an easy, generalized example where 320kbps would make a lot of sense is one of those old concerts from the 80s and 90s that were often recorded in stereo. I have a 1.2GB FLAC track right now that encodes to 526MB at 384kbps, and 351MB at 256kbps. So if I want to save space at virtually no quality loss, 320/384kbps seems like a no-brainer compromise.

Hope I'm making some sense. By the way, what did you think of the idea of separating Jobs folder from Temp Files, to allow redirecting temporary "video.265" output to a RAM drive? I don't think it would have a noticeable effect on encoding times, but it would definitely make the lives of our SSDs easier.

[guest]

I'm chasing some annoying, random drop out's...some days I have no problems, and other days, it's a PITA mess, and I haven't changed anything.

What does this mean :-

[2019-12-03 15:57:53] TCPClient7 raised exception while attempting to sent command.(Not Connected)
[2019-12-03 15:57:53] TCPClient8 raised exception while attempting to sent command.(Not Connected)

I also got this :-

Invalid pointer operation

(Doubt they are related, though.)

[GZZ]

I'm chasing some annoying, random drop out's...some days I have no problems, and other days, it's a PITA mess, and I haven't changed anything.

What does this mean :-

[2019-12-03 15:57:53] TCPClient7 raised exception while attempting to sent command.(Not Connected)
[2019-12-03 15:57:53] TCPClient8 raised exception while attempting to sent command.(Not Connected)

I also got this :-

Invalid pointer operation

(Doubt they are related, though.)

Im thinking you are losing connection between client and server. Could be network related issue or bad network card. Check system log for warnings or errors on the network card and driver. Only use cable, not wifi as it might dropout.

[guest]

Im thinking you are losing connection between client and server. Could be network related issue or bad network card. Check system log for warnings or errors on the network card and driver. Only use cable, not wifi as it might dropout.

Well, it's definitely a network problem....

All the NIC's are onboard (built in, but still could be faulty.)

All the cables are pretty new, (again, they could be faulty.)

No WiFi !!

The weird thing about it is, one day I can have an almost faultless encoding "session", and other days are just crap, and I don't get much done :(, and the fact that I haven't changed anything, adds to the frustration.

[GZZ]

Well, it's definitely a network problem....

All the NIC's are onboard (built in, but still could be faulty.)

All the cables are pretty new, (again, they could be faulty.)

No WiFi !!

The weird thing about it is, one day I can have an almost faultless encoding "session", and other days are just crap, and I don't get much done :(, and the fact that I haven't changed anything, adds to the frustration.

Could be noise from other electrical units if you have bad shielded cable. Try running network at 100mbit and see if it makes a difference. You could also do a ping <ip> -t >> c:\ip.log to see if ping fails doing encoding.

[guest]

Could be noise from other electrical units if you have bad shielded cable. Try running network at 100mbit and see if it makes a difference. You could also do a ping <ip> -t >> c:\ip.log to see if ping fails doing encoding.

Yes, it could be noise, but when its the same equipment running on a good day, and the next day is crap.

I have my main pc's connected to a 24 port switch, using those flat Cat 6 cables (admittedly I have had some trouble with some), then that switch is connected to an 8 port switch in another room, using your "run of the mill" blue round cable, then from that 8 port to the server pc, using flat cable again.

I will give setting then down to 100Mb a try, and because it's so random, a ping would probably not detect a problem.

The connection failures happen randomly during the encode, if you keep an eye on the Encoding Client, a "chunk" will just stop processing, if you stop it, more often than not is will start and complete without issue.

Frustrating to diagnose :(

[Ryushin]

Hey, I thought I'd grab these as well, to see what you'd done...

OMFG, that is SO much better, that would be worth a long encode, for that transformation...well done !!!

BTW, how long did it take to encode the MDegrain3 sample ??

I didn't encode this sample. I cut the same length from the completed file that I did around this time last year. As for time, I think it was around 24 hours with my setup. Since I'm meticulous, I probably ran Blade Runner through a half of dozen times try different settings. So for me it was a week. :)

Here is a post I did when I was just learning MDegrain and doing test encodes and I posted the results of some of my tests:
https://forum.doom9.org/showthread.php?p=1803976#post1803976

[guest]

I didn't encode this sample. I cut the same length from the completed file that I did around this time last year. As for time, I think it was around 24 hours with my setup. Since I'm meticulous, I probably ran Blade Runner through a half of dozen times try different settings. So for me it was a week. :)

Here is a post I did when I was just learning MDegrain and doing test encodes and I posted the results of some of my tests:
https://forum.doom9.org/showthread.php?p=1803976#post1803976

Interesting.

So how come the frame rate is so high ??

edit:- so as you are a regular user of MDegrain filters, (and I'm sure you're using DE), does it take a long time for each chunk to start processing ???

[slalom]

Hey, I thought I'd grab these as well, to see what you'd done...

OMFG, that is SO much better, that would be worth a long encode, for that transformation...well done !!!

edit:- So like I said, I downloaded these to have a look for myself, and I also decided to "test" the (new to me) MDegrain scripts that Atak kindly provided, and I loaded the file 3 times, each with a different MDegrain script, and then when I went to encode, it started to process the chunks and the it errored, stating it couldn't find something in the job folder.....so I checked it Mediainfo, and was surprised to see the frame rate

Video
ID : 1
ID in the original source medium : 4113 (0x1011)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 7 min 24 s
Bit rate : 50.0 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 380.645 FPS
Original frame rate : 23.976 (24000/1001) FPS

So the only thing I could think of was to run it thru Handbrake to bring it down a touch (60fps), loaded it back into RB, away we go...


BTW, how long did it take to encode the MDegrain3 sample ??
Why didn't you use Avisynth -> Output speed?

[slalom]

Added both 1 & 3


??
I found the new scripts in custom script instead of Denoise, is that OK?

Do I need to put those filters on every PC, or just the main?

I get errors on every job with the custom filter

[guest]

Why didn't you use Avisynth -> Output speed?

Hi slalom,

I think you misunderstood this post....that frame rate was the original file, not the one(s) that I ended up with, as I ran out of patience, and didn't complete any of them :(

[guest]

Do I need to put those filters on every PC, or just the main?

I get errors on every job with the custom filter

Hi again, slalom,

I would doubt that you have to add these to every pc, ONLY the pc(s) that you queue up jobs on.

Servers just "help".

So what kind of error are you getting, as I have just gone thru the same process...initialy I had an error due to the name I gave the filters.avs.

[ReinerSchweinlin]

@Ripman
I´ve been where you are - trying to achieve beste quality while encoding - often getting quite high bitrates and sizes...
After many years of listening and comparing - today I simply do it this way:
If for any reason, the quality hast to be preserved a 100%, I simply do not re-encode and jusr mux the original Audio :)
For all other cases, the given presets are sufficient (in my opinion, maybe with age the ears don't pick up enough detail...:D ).

In rare cases where I really need something different than the ripbot presets, I turn to another tool and encode audio there (dmmediaencoder,hybrid,staxrip,etc...) and remux afterwards.

After all, when going for max quality/size ratio, the topic becomes quite subjective and you actualy have to listen to what you encoded intensively.. AAC-LC can be done in VBR, AAC_HE Multichannel can`t, some encoders sound better than others with different source materials, some scale better between channels...

your idea of having a ram-drive as temp is interesting - do you really get a relevant speed advantage over a SSD? Does it make a big difference if you encode for 10 hours and then save 1 minute ssd/ramdisk? Or is the speed difference more noticable?

[ReinerSchweinlin]

Do I need to put those filters on every PC, or just the main?

I get errors on every job with the custom filter
I don´t think so, since the main pc is compiling the scripts and simply servs them via the shared dir to the servers...
Other wise, the server on the main pc should run ok, all others not..
Since errors are everywhere, maybe there is a typo in the filters?
Do the jobs without the custom filter run smoothly?

[slalom]

MDegrain1.avs
#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
video=MDegrain1(video,super,bv1,fv1,thSAD=400)

MDegrain3.avs
#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
bv2=MAnalyse(super,blksize=8,isb=true,delta=2,overlap=4)
bv3=MAnalyse(super,blksize=8,isb=true,delta=3,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
fv2=MAnalyse(super,blksize=8,isb=false,delta=2,overlap=4)
fv3=MAnalyse(super,blksize=8,isb=false,delta=3,overlap=4)
video=MDegrain3(video,super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)

Saved at Tools\AviSynth plugins\Scripts\Custom

Still don't know what this is
Just make sure path in LoadPlugin is correct

The error is general
Error: The file 'E:\Temp\RipBot264temp\job204\video.264' could not be opened for reading: open file error.
-------------------------

Elapsed Time: 00h:03m:53s

[Atak_Snajpera]

If servers encode chunks then problem is not in custom script.

[Ryushin]

For those using MDegrain3, don't forget to add MDegrain3 to the list of multiple processing threads in the Main Settings window tab.

[slalom]

For those using MDegrain3, don't forget to add MDegrain3 to the list of multiple processing threads in the Main Settings window tab.
Same for MDegrain1, right?

[slalom]

I don´t think so, since the main pc is compiling the scripts and simply servs them via the shared dir to the servers...
Other wise, the server on the main pc should run ok, all others not..
Since errors are everywhere, maybe there is a typo in the filters?
Do the jobs without the custom filter run smoothly?
I changed to mdegrain2 and it started

Can someone write the steps to add the extra filters and how to use them please?

[byteshare]

For those using MDegrain3, don't forget to add MDegrain3 to the list of multiple processing threads in the Main Settings window tab.
Easier to just uncheck it IMO. I haven't had problems that I've noticed at least.

Can someone write the steps to add the extra filters and how to use them please?
There are two methods I know:
Method A: Copy/paste(or type) your script into the #Custom section
Method B: Save them as a .avs file in *RipBot\Tools\AviSynth plugins\Scripts\Custom, then select the custom script in the AVS settings.
If you have custom filters (IE not included with RipBot) I've found it easier to have them saved in the same location on all of the machines or you need to write your script to use the network path of the RipBot temp Tools folder (assuming you've saved them in the Tools folder of the main RipBot machine).

If you want more in-depth than that it gets quickly into the subject of how to use AVS, which is more likely IMO solved with specific questions or looking a guides such as: http://avisynth.nl/index.php/Getting_started

[Ryushin]

Easier to just uncheck it IMO. I haven't had problems that I've noticed at least.


Atak, what are you thoughts on on limiting threads to specific filters? What circumstances should they limited or not limited.

[Atak_Snajpera]

Mt should be used only If you do some heavy filtering. If you just resize then using prefetch threads may even decrease performance.

[slalom]

Method B: Save them as a .avs file in *RipBot\Tools\AviSynth plugins\Scripts\Custom, then select the custom script in the AVS settings.
If you have custom filters (IE not included with RipBot) I've found it easier to have them saved in the same location on all of the machines or you need to write your script to use the network path of the RipBot temp Tools folder (assuming you've saved them in the Tools folder of the main RipBot machine).
I used that method with the avs files. I'll copy them now to the other PCs before adding new jobs

[byteshare]

Mt should be used only If you do some heavy filtering. If you just resize then using prefetch threads may even decrease performance.
Ah, well that explains it. I'm always using some filters that are heavy.

[Ronski]

I currently have a 2nd drive for ripbot encodes. What drive type do you suggest as a 2nd drive?

How about one of these https://hexus.net/tech/reviews/storage/137228-seagate-firecuda-520-pcie-gen-4-ssd/ to go with the 570 board.

[slalom]

I tried a new job. MDegrain1 was used. Clients were starting and stopping on a loop. I don't know what is wrong

[byteshare]

I tried a new job. MDegrain1 was used. Clients were starting and stopping on a loop. I don't know what is wrong
I haven't tried MDegrain1, usually SMDegrain (MD3-4) with KLMeans as the prefilter but when I do use just MEdgrain it is usually 2-4.
Does the video play the preview on the main machine? If not, what error.
What errors are you getting on the clients?
Other ideas if you're waiting for another response:
-Try setting up a sample job on the client machines and see if that can encode as well.
--If they don't work, see if you get an error in the AVIsynth preview video.

[slalom]

There is a message, it says

Cannot load file 'C:/Program Files/RipBot264/Tools/AviSynth plugins/mvtools/mvtools2.dll' Platform returned code 126

(E:\Temp\RipBot264temp\Job332/job332_preview.avs, line 39)

this is line 39
#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
video=MDegrain1(video,super,bv1,fv1,thSAD=400)

and my path of the program is

D:\DirectCD\Program Setup Files\RipBot264v1.25.0

so the path is wrong, I have to edit the path in the plugin, right?

missed thet part completely when creating the custom avs files (1 & 2)

[guest]

There is a message, it says

Cannot load file 'C:/Program Files/RipBot264/Tools/AviSynth plugins/mvtools/mvtools2.dll' Platform returned code 126

(E:\Temp\RipBot264temp\Job332/job332_preview.avs, line 39)

this is line 39
#Denoise
Loadplugin("C:\Program Files\RipBot264\Tools\AviSynth plugins\mvtools\mvtools2.dll")
super=MSuper(video,pel=2)
bv1=MAnalyse(super,blksize=8,isb=true,delta=1,overlap=4)
fv1=MAnalyse(super,blksize=8,isb=false,delta=1,overlap=4)
video=MDegrain1(video,super,bv1,fv1,thSAD=400)

and my path of the program is

D:\DirectCD\Program Setup Files\RipBot264v1.25.0

so the path is wrong, I have to edit the path in the plugin, right?

missed thet part completely when creating the custom avs files (1 & 2)

Hi slalom,

Here's my simple way of checking what the path on your setup is..

So, open any job in your queue.

Chose the Avisynth "button"

Click on the "Show Video Script" at the top left side of the little preview screen.

Scroll down 'til you see #Denoise (it might be empty)

Then at the "Denoise" options list, chose "MDegrain2", this will instantly show all the info under #Denoise, and there's the plugin path you have to use....

I hope that works for you...

[guest]

Ah, well that explains it. I'm always using some filters that are heavy.

So, if I create these extra "Custom" MDegrain .avs filters, is it as simple as adding (whatever you've named them), into the "Limit to following filters only" window, and the MT function will be enabled.

[slalom]

It's easy, I have to replace C:/Program Files/RipBot264 in the script, with D:\DirectCD\Program Setup Files\RipBot264v1.25.0

and I think I will be alright

EDIT: Preview Script works now

[guest]

it's easy, i have to replace c:/program files/ripbot264 in the script, with d:\directcd\program setup files\ripbot264v1.25.0

and i think i will be alright

edit: Preview script works now

Excellent. :D:

[Atak_Snajpera]

It's easy, I have to replace C:/Program Files/RipBot264 in the script, with D:\DirectCD\Program Setup Files\RipBot264v1.25.0

and I think I will be alright

EDIT: Preview Script works now

Told you ;)
https://forum.doom9.org/showthread.php?p=1891906#post1891906

[slalom]

Told you ;)
https://forum.doom9.org/showthread.php?p=1891906#post1891906
I just figured that out :readguid:

So, now that everything works, can you add those 2 variations in the same drop down list?

I've been checking some new jobs today, and I had a dilemma, use 1 or 2, or use 2 or 3?

I had to de-select one then press the arrow, and select the other one, preview, too many clicks

I think they have to be in the same menu

[byteshare]

So, if I create these extra "Custom" MDegrain .avs filters, is it as simple as adding (whatever you've named them), into the "Limit to following filters only" window, and the MT function will be enabled.
You have to add the name of the command to the "Limit to following filters only", such as MDegrain1, MDegrain3, etc... It is the name (command) you use in the AVS script to call the function (filter). Not the name of the custom script or the name of the DLL/AVSI/AVS
So, now that everything works, can you add those 2 variations in the same drop down list?
I've been checking some new jobs today, and I had a dilemma, use 1 or 2, or use 2 or 3?
I had to de-select one then press the arrow, and select the other one, preview, too many clicks
I think they have to be in the same menu
If you mean can you have more than one custom script in the drop down, yes. However, if you mean select more than one script at a time, no. If you have functions that you want a the same time you'd have to make a new custom script...even for tiny changes.
Usually what I do if I have a lot of jobs is either add them in groups with the custom filters I want or if only a few jobs add them all at once and then go back and change just those jobs manually.
Ffvideosource is not frame accurate...
When you say frame accurate you don't mean timing, but rather frames completely just get skipped when they are not "accurate"?
I haven't played with GPU decoding much but if that is more accurate than LSmash I'd use that on the machines I can even if sometimes slower.

[slalom]

If you mean can you have more than one custom script in the drop down, yes. However, if you mean select more than one script at a time, no. If you have functions that you want a the same time you'd have to make a new custom script...even for tiny changes.
Usually what I do if I have a lot of jobs is either add them in groups with the custom filters I want or if only a few jobs add them all at once and then go back and change just those jobs manually.
No, just to put all MDegrain variations in the Denoise menu to be easier to compare outputs

[Atak_Snajpera]

No, just to put all MDegrain variations in the Denoise menu to be easier to compare outputs

But you can do the same in custom section.

When you say frame accurate you don't mean timing, but rather frames completely just get skipped when they are not "accurate"?
When I say frame accurate, I mean that If I request frame 100 (with Trim function in script), I get frame 100. With ffms2 ,on other hand, you may get frame 99 or 101 or even totally corrupted frame.

I haven't played with GPU decoding much but if that is more accurate than LSmash I'd use that on the machines I can even if sometimes slower.
GPU decoding is done via LSmash so results will be the same.

[slalom]

But you can do the same in custom section.
Yes, but it's not easy to compare between 1 & 2, or between 2 & 3, as they are in different screens

[Atak_Snajpera]

Yes, but it's not easy to compare between 1 & 2, or between 2 & 3, as they are in different screens

Add MDegrain2 to custom section and you have all 3 in the same spot.

[tkwou]

Hi Atak, I recently updated to v1.25.0. I just tried encoding the movie Life in DE mode and it failed with about 8 minutes left in the movie. When I look at the Chunks folder there are no *.264 files for the last 8 chunks. When I try to run each .cmd file manually I get these errors.

y4m[error]: bad sequence header magic
x2464[error]: could not open input file `_'

If I rerun the .cmd files for any of the completed chunks I get the normal .264 output. I don't understand what's going on. Should I just try recoding again from scratch?

[stax76]

This might be corrupted source file, bug in source filter or broken hardware or driver.

[byteshare]

When I say frame accurate, I mean that If I request frame 100 (with Trim function in script), I get frame 100. With ffms2 ,on other hand, you may get frame 99 or 101 or even totally corrupted frame.
GPU decoding is done via LSmash so results will be the same.
Okay, thank you for clarifying. The only time you use FFMS2 is HEVC, right? I recall thinking it was MPEG2 but you saying it is best with LSmash.

[Atak_Snajpera]

Hi Atak, I recently updated to v1.25.0. I just tried encoding the movie Life in DE mode and it failed with about 8 minutes left in the movie. When I look at the Chunks folder there are no *.264 files for the last 8 chunks. When I try to run each .cmd file manually I get these errors.

y4m[error]: bad sequence header magic
x2464[error]: could not open input file `_'

If I rerun the .cmd files for any of the completed chunks I get the normal .264 output. I don't understand what's going on. Should I just try recoding again from scratch?

open [chunk_number].avs in mpc-hc and see if there is some error message.

[Atak_Snajpera]

Okay, thank you for clarifying. The only time you use FFMS2 is HEVC, right? I recall thinking it was MPEG2 but you saying it is best with LSmash.

With latest version of lsmash I recommend to use it for ALL formats! Just forget about ffms. Lsmash is superior in almost every aspect.

[slalom]

Add MDegrain2 to custom section and you have all 3 in the same spot.
That's a good idea, just did!

[tkwou]

Thanks Atak, I just tried restarting from Ripbot264 and DE just finished the remaining 8 minutes.

[jlpsvk]

With latest version of lsmash I recommend to use it for ALL formats! Just forget about ffms. Lsmash is superior in almost every aspect.

can you be more concrete? :) still using ffms. :)

[guest]

You have to add the name of the command to the "Limit to following filters only", such as MDegrain1, MDegrain3, etc... It is the name (command) you use in the AVS script to call the function (filter). Not the name of the custom script or the name of the DLL/AVSI/AVS

Thanks heaps for this, byteshare :)

[guest]

So I would imagine that most of you have a reasonable collection of both unprocessed and processed movies & series, etc, which can add up to a LOT of Terabytes.

I would be interested to know what some of you (who are happy to oblige) have & use for storage, of said data ???

I have several reasonable capacity Supermicro storage servers, and a couple of Unraid "box's", but unfortunately, every so often there is a problem, and as a result, I get data corruption.

I don't really have too much backed up, so sometimes it's a complete loss :(, which can be very frustrating, especially if it was irreplaceable.

Anyway, I'm just curious...

[GZZ]

So I would imagine that most of you have a reasonable collection of both unprocessed and processed movies & series, etc, which can add up to a LOT of Terabytes.

I would be interested to know what some of you (who are happy to oblige) have & use for storage, of said data ???

I have several reasonable capacity Supermicro storage servers, and a couple of Unraid "box's", but unfortunately, every so often there is a problem, and as a result, I get data corruption.

I don't really have too much backed up, so sometimes it's a complete loss :(, which can be very frustrating, especially if it was irreplaceable.

Anyway, I'm just curious...


Should be in another thread or properly another group, try search forum as its out off topic for this thread.

[tkwou]

Hi Atak, I recently updated to v1.25.0. I just tried encoding the movie Life in DE mode and it failed with about 8 minutes left in the movie. When I look at the Chunks folder there are no *.264 files for the last 8 chunks. When I try to run each .cmd file manually I get these errors.

y4m[error]: bad sequence header magic
x2464[error]: could not open input file `_'

If I rerun the .cmd files for any of the completed chunks I get the normal .264 output. I don't understand what's going on. Should I just try recoding again from scratch?

Hi Atak, strange this just happened again. Processing stopped with just 2 chunks to go. With the missing chunks, the CombineChunk.cmd fails and video.264 is missing for the mux. I restarted the encode and again it finished the final 2 chunks and everything completed fine.

[Atak_Snajpera]

Hi Atak, strange this just happened again. Processing stopped with just 2 chunks to go. With the missing chunks, the CombineChunk.cmd fails and video.264 is missing for the mux. I restarted the encode and again it finished the final 2 chunks and everything completed fine.

You have some network issues or some instability in RAM/CPU...

[Ryushin]

With latest version of lsmash I recommend to use it for ALL formats! Just forget about ffms. Lsmash is superior in almost every aspect.

So the random frames happening around the end of the DE chunks should be fixed? Works for VC-1 too?

[Atak_Snajpera]

So the random frames happening around the end of the DE chunks should be fixed? Works for VC-1 too?

yes...

[guest]

Should be in another thread or properly another group, try search forum as its out off topic for this thread.

OMG, GZZ !!!

I knew that I was off topic for this thread, but I thought I'd just ask the question of the member's on THIS thread, as they MUST have some form of reliable storage.

So why you reply to state the bleeding obvious is a waste of your time, when for the same effort you could have said something like ....I have a Synology or Unraid backup....

[guest]

Is there a way to confirm that these "Custom" MDegrain filters are / or have been used ??
Other than creating some short test encodes and comparing the fps, kbps (whilst processing), or time taken to encode & final file size ??

Anyway, I did that, and here's some interesting results:-

a test tv episode, mp4, 17 minutes long.

MDegrain1 took 8:34m

MDegrain2 took 11:43m

MDegrain3 took 15:18m

they are ALL with MT enabled.

Then I ran MDegrain1 test again changing the MT settings.

with no MT, it took a "massive" 28:25m.

with MT enable, but using 16 threads took the original 8:34m

with 8 threads it took 8:17m

with 0 (which must be unlimited) it took 8:08m (on my 24 thread pc)

so going on that all the above times would be a touch faster if I had of had it set on 0 threads.

As for file size change:-

original file is 255,856kb @ 1,923kb/s

MD1 encoded is 303,196kb @ 2,192kb/s

MD2 encoded is 289,490kb @ 2,126kb/s

MD3 encoded is 281,275kb @ 2,126kb/s

No other filters or changes where made, using CRF 20, and ONLY the custom MDegrain filters.

[Atak_Snajpera]

Zero means number of physical cores If I remember correctly. So If your CPU is 12c/24t then value will be 12.
Regarding file size. You should try with noisy source. Difference will be much higher. MD2 should give you up to 1.8x bitrate reduction.

[guest]

Zero means number of physical cores If I remember correctly. So If your CPU is 12c/24t then value will be 12.
Regarding file size. You should try with noisy source. Difference will be much higher. MD2 should give you up to 1.8x bitrate reduction.

So does "0" automatically determine how many cores/threads are available ??

As "0" is the fastest of the settings I tried.

Also, I DID do a test on much noisier source (The Fifth Element), and there was a bigger file size/bitrate difference between the three.

[guest]

Me again,

I thought I'd run a few job thru RB today, on a pc that I was using successfully just last week, and I just keep getting errors, several time during the same job.

Cannot find file F:\Temp\RipBot264temp\job10\video.264
Error importing F:\Temp\RipBot264temp\job10\video.264:name=The Report 2019: Requested URL is not valid or cannot be found

It got thru approx 12 chunks before the 1st error, then after restarting the job, it got thru to chunk 24, then the next time I think it got thru quite a bit further, and errored again.

After several more attempts with different jobs, restarting the pc, I gave up, and went and did some cooking.

[Atak_Snajpera]

Auto-update 2019.12.09
Added: EncodingClient combines chunks with output file on fly as soon as they are encoded in correct order
Added: Audio profiles in Profile\Audio.txt
Changed: Jobless servers will be suspended only if ETA>60s (reason: reduce unnecessary suspend -> resume cycles)
Fixed: Excessive flickering of encoding status in EncodingClient
Fixed: EncodingClient didn't resume suspended jobless servers before starting second pass.
Fixed: EncodingServer didn't go to suspend mode again after previous resume

[ReinerSchweinlin]

Thanks for the update log. This is very helpful, please continue posting when updates are done :)

[byteshare]

So I would imagine that most of you have a reasonable collection of both unprocessed and processed movies & series, etc, which can add up to a LOT of Terabytes.

I would be interested to know what some of you (who are happy to oblige) have & use for storage, of said data ???
https://www.reddit.com/r/DataHoarder/
I use 1 to 1 backups

Is there a way to confirm that these "Custom" MDegrain filters are / or have been used ??
Other than creating some short test encodes and comparing the fps, kbps (whilst processing), or time taken to encode & final file size ??
Won't give you the benefits of a test encode (seeing the actual file results) but you can check:
(on a job) edit > AviSynth > Show Video Script > Look under the custom section that you see your script, then...
> Preview (first to make sure there are no errors in your script.
If no errors and your script is being loaded > AVS Meter to see the speed and such it runs at.
If you're still skeptical that your script isn't being used because of a mistake in your code that you're not seeing try removing it and then running the AVS Meter again, if there is no difference in speed (FPS) then something could be wrong with your script and then you'd want to run a test encode to see if there are any filesize differences, if none then it is likely your script or you're using a filter that is not having an impact either by design or by an error in your script.
(tip, on your script be sure "video" is somewhere after the "=" otherwise it isn't being using in the final video sent to the HEVC encoder, otherwise it is fine if you're using it for a mask, prefilter, etc...

original file is 255,856kb @ 1,923kb/s
MD1 encoded is 303,196kb @ 2,192kb/s
MD2 encoded is 289,490kb @ 2,126kb/s
MD3 encoded is 281,275kb @ 2,126kb/s
Besides MD1-3 (there are higher ones as well) you can try adjusting the strength as well (thSAD, higher values mean more removal of grain/noise but too high removes noticeable details). Can also try after MD a filter like FineDehalo/HQDeringmod (there are a lot of choices in this area) to remove some created/leftover temporal noise (mosquitoes) depending on your source and filter results.

[guest]

Auto-update 2019.12.09
Added: EncodingClient combines chunks with output file on fly as soon as they are encoded in correct order
Added: Audio profiles in Profile\Audio.txt
Changed: Jobless servers will be suspended only if ETA>60s (reason: reduce unnecessary suspend -> resume cycles)
Fixed: Excessive flickering of encoding status in EncodingClient
Fixed: EncodingClient didn't resume suspended jobless servers before starting second pass.
Fixed: EncodingServer didn't go to suspend mode again after previous resume

Thanks very much for that, will be a lot easier on the eyes :)

And how do you use the "audio.txt" ?

Just need to add that I updated a client & 1 server with the latest, and it was running well, it had done 2 3/4 jobs, and Encoding Client just stopped all processing, both Client & Server...had to force RB shutdown with Processhacker, started again, and away it went.

edit addition:-since that error, I have had no more problems, maybe it just needed to "warm up"

I do have a question about the function of the Encoding Client "button" on the very right of the 4 buttons...

Not sure what you've named it, but I did a little fiddling, and noticed that if that button is enabled, as soon as the processing chunk is complete, that server/port goes idle....and if thats the only chunk on a particular server, it would then shut that server down....so my question is why, if there are more chunks for that server to process.

All that comes to my old mind is that if you had a very long queue and you wanted to stop it part way thru, you'd enable that button. Then eventually all the servers would turn themselves off, and you'd only be left with the client.

I actually thought that that button would do similar to the suspend option, BUT turn them off !!

I really like the suspend option, that's great, and seems to work well for me. :)

[guest]

https://www.reddit.com/r/DataHoarder/
I use 1 to 1 backups


Won't give you the benefits of a test encode (seeing the actual file results) but you can check:
(on a job) edit > AviSynth > Show Video Script > Look under the custom section that you see your script, then...
> Preview (first to make sure there are no errors in your script.
If no errors and your script is being loaded > AVS Meter to see the speed and such it runs at.
If you're still skeptical that your script isn't being used because of a mistake in your code that you're not seeing try removing it and then running the AVS Meter again, if there is no difference in speed (FPS) then something could be wrong with your script and then you'd want to run a test encode to see if there are any filesize differences, if none then it is likely your script or you're using a filter that is not having an impact either by design or by an error in your script.
(tip, on your script be sure "video" is somewhere after the "=" otherwise it isn't being using in the final video sent to the HEVC encoder, otherwise it is fine if you're using it for a mask, prefilter, etc...


Besides MD1-3 (there are higher ones as well) you can try adjusting the strength as well (thSAD, higher values mean more removal of grain/noise but too high removes noticeable details). Can also try after MD a filter like FineDehalo/HQDeringmod (there are a lot of choices in this area) to remove some created/leftover temporal noise (mosquitoes) depending on your source and filter results.

Thanks for the great reply :)

I did do a run on the first few minutes of The Fifth Element (which is VERY noisey), and I was pretty happy the MD3 results (thSAD=800).

I do have slightly different thSAD numbers for MD1 & MD2.

I might go back to a much older version of RB, and see what the HQDN3D settings were, and maybe create a custom filter for non 10bit encodes.

[Ryushin]

Besides MD1-3 (there are higher ones as well) you can try adjusting the strength as well (thSAD, higher values mean more removal of grain/noise but too high removes noticeable details). Can also try after MD a filter like FineDehalo/HQDeringmod (there are a lot of choices in this area) to remove some created/leftover temporal noise (mosquitoes) depending on your source and filter results.

Do you have a copy of those custom scripts for FineDehalo/HQDeringmod. Originally that is what I was using mdegrain2+hqdn3d. If it does a good job I'm hoping Atak will add those as additional features.

[Ryushin]

Just updated and I noticed something, which I'm not sure is related to the last update or not. If I open RipBot264, change the order of the encodes, close RipBot264 and open it again, the order goes back to what it was before I changed the order.

[Atak_Snajpera]

Thanks very much for that, will be a lot easier on the eyes :)

And how do you use the "audio.txt" ?

Just need to add that I updated a client & 1 server with the latest, and it was running well, it had done 2 3/4 jobs, and Encoding Client just stopped all processing, both Client & Server...had to force RB shutdown with Processhacker, started again, and away it went.

edit addition:-since that error, I have had no more problems, maybe it just needed to "warm up"

I do have a question about the function of the Encoding Client "button" on the very right of the 4 buttons...

Not sure what you've named it, but I did a little fiddling, and noticed that if that button is enabled, as soon as the processing chunk is complete, that server/port goes idle....and if thats the only chunk on a particular server, it would then shut that server down....so my question is why, if there are more chunks for that server to process.

All that comes to my old mind is that if you had a very long queue and you wanted to stop it part way thru, you'd enable that button. Then eventually all the servers would turn themselves off, and you'd only be left with the client.

I actually thought that that button would do similar to the suspend option, BUT turn them off !!

I really like the suspend option, that's great, and seems to work well for me. :)

https://i.postimg.cc/dtqFRTNP/Untitled-1.png = SUSPEND JOBLESS SERVER (All chunks have been already assigned to other servers)

https://i.postimg.cc/nc0J3cNs/Untitled-2.png = SHUTDOWN SERVER AT IDLE STATE (Finish encoding current chunk and then shutdown)

[byteshare]

Do you have a copy of those custom scripts for FineDehalo/HQDeringmod. Originally that is what I was using mdegrain2+hqdn3d. If it does a good job I'm hoping Atak will add those as additional features.
I usually use SMDegrain rather than straight MDegrain. I don't like HQDN3D on its own. I'd rather use KNLMeansCL with GPU, which is actually the prefilter I use with SMDegrain.
For FineDehalo I either use the default: FineDehalo() or something like:
"FineDehalo(rx=2.0,ry=2.0,thmi=80,thma=128,thlimi=50,thlima=100,darkstr=1.0,brightstr=1.0,showmask=0,contra=0.0,excl=true)"
There are some safer settings if you're using anime/cartoons.

For HQDeringmod I also use the defaults "HQDeringmod()" or something like: "HQDeringmod(nrmode=2,Y=3,U=3,V=3,lsb=False,thr=255.0,darkthr=255.0)"

For those filters and others it depends on the source and what you want.

I'd stay away from thSAD values of 800 and higher for MDegrain, as it can soften the image under closer inspection. SMDegrain values of thSAD are a little different because it combines more filters.

[guest]

https://i.postimg.cc/dtqFRTNP/Untitled-1.png = SUSPEND JOBLESS SERVER (All chunks have been already assigned to other servers)

https://i.postimg.cc/nc0J3cNs/Untitled-2.png = SHUTDOWN SERVER AT IDLE STATE (Finish encoding current chunk and then shutdown)

Thankyou for that, I thought that https://i.postimg.cc/dtqFRTNP/Untitled-1.png put that particular server to "sleep" after the encoding was completed, then once the next job queue was starting, they would wake up, and start encoding :o

However, I really can't see that https://i.postimg.cc/nc0J3cNs/Untitled-2.png is all that necessary until towards the very end of the chunk queue, and you want individual servers to turn off.

The only reason I can see for it's use is if you have a LONG queue, and you want to stop it part way thru (for what ever reason).

I think one nice function would be to have a similar option for the client, but when the chunk that was processing completed, it didn't start the next one.

Used in conjunction with https://i.postimg.cc/nc0J3cNs/Untitled-2.png you could completely (automatically) pause or stop an encode, without doing it manually.

Anyway, enough of that, it's all a little bit over my head, but I know what works for me.

Could you please explain the use of the new "audio.txt" ?

[guest]

I usually use SMDegrain rather than straight MDegrain. I don't like HQDN3D on its own. I'd rather use KNLMeansCL with GPU, which is actually the prefilter I use with SMDegrain.
For FineDehalo I either use the default: FineDehalo() or something like:
"FineDehalo(rx=2.0,ry=2.0,thmi=80,thma=128,thlimi=50,thlima=100,darkstr=1.0,brightstr=1.0,showmask=0,contra=0.0,excl=true)"
There are some safer settings if you're using anime/cartoons.

For HQDeringmod I also use the defaults "HQDeringmod()" or something like: "HQDeringmod(nrmode=2,Y=3,U=3,V=3,lsb=False,thr=255.0,darkthr=255.0)"

For those filters and others it depends on the source and what you want.

I'd stay away from thSAD values of 800 and higher for MDegrain, as it can soften the image under closer inspection. SMDegrain values of thSAD are a little different because it combines more filters.

So how would you set these up a a Custom filter ??

[guest]

I have had a pretty frustrating couple of days with the problem that is shown in the screen shots (attached).

https://www.mediafire.com/file/dxbozrqmj3algb9/Client_encoding_problems.rar/file

I have a job encoding as it should, using a couple of servers, and it gets to a random point, and everything just stops (as far as the Encoding Client is concerned), if you check cpu usage, and TCP traffic, etc, it's still encoding in the background, but no progress is shown, it's completes the chunk that it was on, (when it locked up), then it stops, will not start another chunk.

The abort function does not work, you have to "kill" it with processhacker or similar, then once RB is finally shutdown...if you start that same job again, nothing starts :(

I know a video capture would have been preferable, but I had nothing set up on the pc to do that.

So the pic's (in order) shows that the Client is stalled, but the Server is still processing, in the 2nd pic it shows that Client is still the same, but the Server has now finished.
Then 3rd pic is restarting the job...and that's where it stays.

This has happened on 2 different pc's, and different jobs, all freshly loaded using the very latest RB version.

[Atak_Snajpera]

Can you move encoding client window or IT is locked with not responding message in title?

[guest]

Can you move encoding client window or IT is locked with not responding message in title?

Not 100% what you mean, but I don't think I tried to move it, but I'm pretty sure you couldn't minimize it, close it, so maybe if I had of tried to move it, it wouldn't have....I don't recall any message displayed.

[duffbeer]

I have a feature request.
The file currently encoding has an eta which is really useful. I would also like to know the estimated file size.
I realise that it may not be 100% accurate and that the predicted size will change during the encode but it would be useful none the less.

[Ryushin]

Can you move encoding client window or IT is locked with not responding message in title?

I'll try that today. I noticed this happening after the last update. Also getting jobs that are failing with ERROR in queue list. The job never finishes and I assume it's crashing or hanging and it's dumping a log in the output directory with:
Error: The file 'D:\Temp\RipBot264temp\job2\video.265' could not be opened for reading: open file error.

Probably related to the new code that is combing chunks during the encoding job.

[forkicks]

I have had a pretty frustrating couple of days with the problem that is shown in the screen shots (attached).

https://www.mediafire.com/file/dxbozrqmj3algb9/Client_encoding_problems.rar/file

I have a job encoding as it should, using a couple of servers, and it gets to a random point, and everything just stops (as far as the Encoding Client is concerned), if you check cpu usage, and TCP traffic, etc, it's still encoding in the background, but no progress is shown, it's completes the chunk that it was on, (when it locked up), then it stops, will not start another chunk.

The abort function does not work, you have to "kill" it with processhacker or similar, then once RB is finally shutdown...if you start that same job again, nothing starts :(

I know a video capture would have been preferable, but I had nothing set up on the pc to do that.

So the pic's (in order) shows that the Client is stalled, but the Server is still processing, in the 2nd pic it shows that Client is still the same, but the Server has now finished.
Then 3rd pic is restarting the job...and that's where it stays.

This has happened on 2 different pc's, and different jobs, all freshly loaded using the very latest RB version.

I can confirm this behaviour with latest updates (updated today). Main RB window completely stops responding, encoder window controls react but do nothing (ie, buttons are depressed, but no action occurs).

fK

[jfisher1740]

I have had a pretty frustrating couple of days with the problem that is shown in the screen shots (attached).

https://www.mediafire.com/file/dxbozrqmj3algb9/Client_encoding_problems.rar/file

I have a job encoding as it should, using a couple of servers, and it gets to a random point, and everything just stops (as far as the Encoding Client is concerned), if you check cpu usage, and TCP traffic, etc, it's still encoding in the background, but no progress is shown, it's completes the chunk that it was on, (when it locked up), then it stops, will not start another chunk.

The abort function does not work, you have to "kill" it with processhacker or similar, then once RB is finally shutdown...if you start that same job again, nothing starts :(

I know a video capture would have been preferable, but I had nothing set up on the pc to do that.

So the pic's (in order) shows that the Client is stalled, but the Server is still processing, in the 2nd pic it shows that Client is still the same, but the Server has now finished.
Then 3rd pic is restarting the job...and that's where it stays.

This has happened on 2 different pc's, and different jobs, all freshly loaded using the very latest RB version.

+1 to this same issue and behavior for DE. It seems like this started after the most recent auto update.

[Atak_Snajpera]

I will prepare debug version for you guys. I will need approximately 1 hour.

[Ryushin]

I'll try that today. I noticed this happening after the last update. Also getting jobs that are failing with ERROR in queue list. The job never finishes and I assume it's crashing or hanging and it's dumping a log in the output directory with:
Error: The file 'D:\Temp\RipBot264temp\job2\video.265' could not be opened for reading: open file error.

Probably related to the new code that is combing chunks during the encoding job.

I could move both windows. But if I click abort, then the Encoding Client window locks up with Aborting in it's title and I cannot move it. Only by killing it in the Task Manager does it make it die.

[Atak_Snajpera]

Use this debug version -> https://www.mediafire.com/file/e4w9w4a4dy3ye0d/EncodingClient.exe/file
and then post content of Debug.txt when encodingclient.exe hangs.

[Ryushin]

Use this debug version ->
and then post content of Debug.txt when encodingclient.exe hangs.

It hung, I clicked abort and waited 15 minutes and it did not abort. Log:
http://ix.io/24aS

[Atak_Snajpera]

It hung, I clicked abort and waited 15 minutes and it did not abort. Log:
http://ix.io/24aS

From log i see that you are experiencing different bug than Paul and others because all chunks have been combined with video.part file.

[guest]

From log i see that you are experiencing different bug than Paul and others because all chunks have been combined with video.part file.

Sorry I haven't responded earlier, but I'm in a very different time zone to most of you....(unless I've woken up in the middle of the night, and checked in)

Anyways...

I got around to doing a test with the same movie I had problems with yesterday, and it's funny you're sitting there waiting for it to f*** up, and it did :(, and it didn't get as far, this time.

Please check out the attached screenshots & debug.txt's.

https://www.mediafire.com/file/8g4yarseb9tdiax/Debug.rar/file

Firstly (attempt 1) I just tried to see what yesterdays stalled job would do the the debug client (not a lot)

Then I reloaded the job, (attempt 2) and it all started OK (a bit slow on the uptake due to using MD2 custom filter)

Anyway, to my untrained eye, there doesn't appear to be any errors, (as per Ryushin's post), it just simply stopped, however the server I had helping, completed the 4 chunks it had left to do, then it stopped, without issue.

But once an Abort was initiated, the window was locked in position (if that means anything), it didn't Abort, so back to ProcessHacker to do the job.

[Atak_Snajpera]

At least i know that code resonsible for combining chunks works as intended. I will prepare debug2 with more loging in other parts of the code.

[guest]

At least i know that code resonsible for combining chunks works as intended. I will prepare debug2 with more loging in other parts of the code.

Well, that's a good thing, 'cause that's a good time saver :)

So, will keep an eye out for debug2, and give it a run.

Are the screen shots helpful ??

Is there any other info you might need ??

[Ryushin]

From log i see that you are experiencing different bug than Paul and others because all chunks have been combined with video.part file.

Here is the debug from the hang that has occurred most of the time:
http://ix.io/24cD

[guest]

Here is the debug from the hang that has occurred most of the time:
http://ix.io/24cD

Did it hang before the encode completed ??

Did it work with the Abort function ??

[Ryushin]

Did it hang before the encode completed ??

Did it work with the Abort function ??

It hung before the encode completed. Abort results in a hard hang for the Encoding Client and must be killed using the Task Manager.

[guest]

It hung before the encode completed. Abort results in a hard hang for the Encoding Client and must be killed using the Task Manager.

Yep, sounds about right.

Well, another interesting challenge for Atak.