RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY [Archive] - Page 273

Viper714

16th June 2015, 20:42

Atak,
So far it does NOT look promising. It is very, very slow as before on Pass 1. The initial start was fast but as it got further into it it started to slow down.

Here is a link to a pdf I put together showing the encoding with the CPU usage of each machine as you requested before. Once the encoding is complete, if it does not crash first, I will post the logs and text files.
https://dl.dropboxusercontent.com/u/45570839/1.18.3%20Beta%20Encoding%20of%20Last%20of%20the%20Mohicans.pdf

Viper714

16th June 2015, 22:17

Atak,
The encoding was successful!! Really surprised as it did not start off very well. It took 02h:05m:47s to complete. Pass 1 usually hangs pretty high in the fps range and Pass 2 in the 6-12 fps range. Not sure why it was different but at least it worked. I will try a few other movies later for verification.

Here are the log files for you to review just in case
https://dl.dropboxusercontent.com/u/45570839/Mohicans2%20Log.zip

Thank you so much for your help in getting this working. Your efforts in supporting this program are so appreciated!!!!

Ronski

17th June 2015, 10:11

Atak_Snajpera

17th June 2015, 10:41

viper
first pass is by default much faster than second because it only analyses footage for optimal bitrate distribution. Actual encoding is done in second pass hence much slower encoding speed.

Viper714

17th June 2015, 15:37

viper
first pass is by default much faster than second because it only analyses footage for optimal bitrate distribution. Actual encoding is done in second pass hence much slower encoding speed.

Atak, I kind of figured that. I may not have detailed my post properly. I normally see that PASS 1 goes very quickly for the reason you stated. However, in the last encoding that I did yesterday (The Last of the Mohicans) I noticed that PASS 1 started out fast but slowed down extremely.

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/RipBot%20Image.png (http://s361.photobucket.com/user/pcubillos/media/Issues/RipBot%20Image.png.html)

Using the above image, this shows PASS 1 as it was almost finished. Notice the RED boxes where the FPS is very low and the GREEN boxes showing the normal FPS range for PASS 1. In the past this would be the normal (GREEN) for PASS 1. Not sure why it would drop to 2 - 3 fps or even 9 fps on a read only. I will do another rip sometime today for additional comparison. I will also try version 1.18.1 just in case...

Sorry if I am being a pest. Just trying to pass information...

Tsusai

17th June 2015, 15:50

Atak,
First findings,

UAC dialog box on start - normal
UAC dialog box second time on start - for server activation
UAC dialog box third time when adding a second server
Smart Screen Prevention box on one of my Windows 8 machines. Clicking OK does not work. Unable to run Ripbot256. http://i361.photobucket.com/albums/oo54/pcubillos/Issues/2015-06-16_13-29-21_1.png (http://s361.photobucket.com/user/pcubillos/media/Issues/2015-06-16_13-29-21_1.png.html)

I turned off Smart Screen and got it to work. Please try to fix this as this should not have to be turned off

Will get back to you after encoding...

Ah SmartScreen. You got 2 things you can do. It's just like Win 7 when you download any file. It flags it from the Internet and just prompts you to make sure you know this.
1) When it prompts, you can click on More Info. It should give you a Run prompt
2) RT Click->Properties on the EXE. You might see an Unblock Button. Press it. You can do this on zip/7z/rar files as well prior to extraction so the extracted files aren't flagged from the Internet as well.

Viper714

17th June 2015, 18:17

TSUSAI: Thank you for this information. I changed it as you mentioned and it worked great!!!!
:thanks:

Atak_Snajpera

17th June 2015, 18:58

Atak, I kind of figured that. I may not have detailed my post properly. I normally see that PASS 1 goes very quickly for the reason you stated. However, in the last encoding that I did yesterday (The Last of the Mohicans) I noticed that PASS 1 started out fast but slowed down extremely.

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/RipBot%20Image.png (http://s361.photobucket.com/user/pcubillos/media/Issues/RipBot%20Image.png.html)

Using the above image, this shows PASS 1 as it was almost finished. Notice the RED boxes where the FPS is very low and the GREEN boxes showing the normal FPS range for PASS 1. In the past this would be the normal (GREEN) for PASS 1. Not sure why it would drop to 2 - 3 fps or even 9 fps on a read only. I will do another rip sometime today for additional comparison. I will also try version 1.18.1 just in case...

Sorry if I am being a pest. Just trying to pass information...

No it is ok. I'm also curious why you have such low cpu usage even with two instances.

I would like you to do the following things

1) Download process explorer https://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

2) Run again encoding but this time show me cpu usage for avs2yuv.exe and x264.exe

example
http://i.cubeupload.com/4KEJpK.png

I predict that low fps is due to low cpu usage caused by something wrong going with your router. Basically avs2yuv.exe is waiting too long for frames from main pc.

Viper714

17th June 2015, 23:51

No it is ok. I'm also curious why you have such low cpu usage even with two instances.

I would like you to do the following things

1) Download process explorer https://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

2) Run again encoding but this time show me cpu usage for avs2yuv.exe and x264.exe

I predict that low fps is due to low cpu usage caused by something wrong going with your router. Basically avs2yuv.exe is waiting too long for frames from main pc.

Atak:

Here is the info you requested...

Last of the Mohicans
http://i361.photobucket.com/albums/oo54/pcubillos/Issues/Encoding%20Client.png (http://s361.photobucket.com/user/pcubillos/media/Issues/Encoding%20Client.png.html)

Process Explorer
http://i361.photobucket.com/albums/oo54/pcubillos/Issues/Process%20Explorer.png (http://s361.photobucket.com/user/pcubillos/media/Issues/Process%20Explorer.png.html)

Here is the same information 5 minutes later

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/Encoding%20Client2.png (http://s361.photobucket.com/user/pcubillos/media/Issues/Encoding%20Client2.png.html)

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/Process%20Explorer2.png (http://s361.photobucket.com/user/pcubillos/media/Issues/Process%20Explorer2.png.html)

Hope that helps!!!

Atak_Snajpera

18th June 2015, 12:03

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/RipBot%20Image.png

Something tells me that your network is not working in 1000 Mbps but in slower 100 Mbps mode.

Just take a look.
According to http://www.blu-ray.com/movies/The-Last-of-the-Mohicans-Blu-ray/10898/ this movie has average bitrate of 38 Mbps. If we subtract audio stream (DTSMA ~4 Mbps) and all subtitles we should get pure video stream bitrate of ~33 Mbps.

Above image shows that your average encoding speed is 73 fps.
So If we divide 73 fps by 24 (movie frame rate) we get ~3.

33 Mbps * 3 = 99 Mbps.

To verify my theory you should do some simple copy test between PCs with some large file. If your copy speed is limited to ~12 MB/s then you have your answer why first pass is slow.

mini-moose

18th June 2015, 15:43

I'm playing around with this interesting tool for the first time.

Got a couple of questions:

1) Why is Sharpen used as default? I tried to remove it from the .ini completely but then I got an error when starting encode saying
Sharpen values are incorrect. Changed it to 0.0 on .ini now, but I'm not sure if that disables it completely.

2) Seems like DSS is always the default decoder unless distributed encoding is used in which case the source is indexed by ffms2.
Is there a way to choose between DSS and indexing for single pc encode?

thanks in advance.

Atak_Snajpera

18th June 2015, 16:18

1) 0 = means disabled. (filter has nothing to do) I've added gentle sharpening because downscaled movie looks better (especially on my ancient PSP with screen 480x272)
2) If ain't broke don't fix it. That's my motto. DirectShowSource has been working well for all those years. It is ok if you do not need frame accurate seeking.

mini-moose

18th June 2015, 17:20

1) 0 = means disabled. (filter has nothing to do)

thanks for confiming. I had a hard time finding documentation on Sharpen for some reason.

2) If ain't broke don't fix it. That's my motto. DirectShowSource has been working well for all those years. It is ok if you do not need frame accurate seeking.

Good motto :) Was just wondering if there was a way to do it differently and I couldn't find it.

Viper714

18th June 2015, 21:28

Something tells me that your network is not working in 1000 Mbps but in slower 100 Mbps mode......

Atak, I will run a LAN Speed test in a few. But just FYI, I decided to swap the Ethernet cable to another plug on the motherboard on my server. I tried the same encoding with 1.18.3 and got the same result on Pass 1. Average was 2-4 FPS.
Then I tried version 1.18.1 and I got 17 - 24 fps range.

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/2015-06-18_16-04-30.png (http://s361.photobucket.com/user/pcubillos/media/Issues/2015-06-18_16-04-30.png.html)

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/2015-06-18_16-11-11.png (http://s361.photobucket.com/user/pcubillos/media/Issues/2015-06-18_16-11-11.png.html)

All was going great up to the end of PASS 1 until Ripbot either crashed of froze.

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/2015-06-18_16-14-09.png (http://s361.photobucket.com/user/pcubillos/media/Issues/2015-06-18_16-14-09.png.html)

I tried turning off the offending server but would not work. THe abort button did work and it started encoding the last chunk in PASS 1. It is continuing as I type at around 24 FPS. IF I had a network issue wouldn't I still get the same issue even in 1.18.1??

Viper714

19th June 2015, 07:29

To verify my theory you should do some simple copy test between PCs with some large file. If your copy speed is limited to ~12 MB/s then you have your answer why first pass is slow.

Atak, Here are my results on a LAN test from the SERVER-PC to the other three computers. I tested a 500 MB file as you will notice.

X-51 PC
http://i361.photobucket.com/albums/oo54/pcubillos/Issues/X-51.png (http://s361.photobucket.com/user/pcubillos/media/Issues/X-51.png.html)

LTPC
http://i361.photobucket.com/albums/oo54/pcubillos/Issues/LTPC.png (http://s361.photobucket.com/user/pcubillos/media/Issues/LTPC.png.html)

PAUL-PC
http://i361.photobucket.com/albums/oo54/pcubillos/Issues/PAUL-PC.png (http://s361.photobucket.com/user/pcubillos/media/Issues/PAUL-PC.png.html)

Hope this helps to answer this conundrum!!!

Atak_Snajpera

19th June 2015, 10:30

My last idea. Copy x264 folder from 1.18.1 version to 1.18.3 (TOOLS/x264)

slalom

19th June 2015, 13:12

I had a similar problem. One RJ45 plug had issues and I replaced it.

So check your plugs for signs of abuse

Viper714

19th June 2015, 16:21

My last idea. Copy x264 folder from 1.18.1 version to 1.18.3 (TOOLS/x264)

Okay I will try it later today. Will let you know
:thanks:

Viper714

19th June 2015, 16:25

I had a similar problem. One RJ45 plug had issues and I replaced it.

So check your plugs for signs of abuse

Humm!! Good point. Amazingly I recently purchased an Ethernet Cable tester. Time to put it to good use!!! Thanks for the recommendation...

slalom

19th June 2015, 17:26

Do an optical inspection. I still had 1 Gbps connection on the Task Manager, but the the cable inside the plug was pulled about 5mm

mini-moose

20th June 2015, 13:15

I have a concern about the distributed encoding feature.

I see it's done by splitting the video encode into small parts (trim in avs) and those get distributed between the computers. I can see how that works ok for CRF encodes (though it might mess up GOPs),
but with 2-pass, doesn't it diminishes the purpose of 1st pass? The video is split into ~10mins segments (maybe more with more pcs, I tried with only 2)with each getting it's own stats file, which means
the video is not analyzed as a whole, and thus bitrate distribution is flawed for 2nd pass. Unless I'm missing/misunderstanding something, that's pretty much as effective as 1-pass ABR.

Atak_Snajpera

20th June 2015, 14:22

Unless I'm missing/misunderstanding something, that's pretty much as effective as 1-pass ABR.
10 min chunk is long enough to be safe. During that period you will have mix of static scenes (less bitrate needed) and dynamic (more bitrate needed). Nevertheless I do not recommend going down to 1min chunk size because like you already mentioned bitrate distribution would have been flattened too much. 10 min chunks size is just a compromise between encoding efficiency and quality.

Viper714

20th June 2015, 21:14

My last idea. Copy x264 folder from 1.18.1 version to 1.18.3 (TOOLS/x264)

Atak,

SUCCESS!!!

That actually did it!!! I had not checked the network cables yet as I wanted to try this last suggestion. Man these are beautiful numbers again!!

http://i361.photobucket.com/albums/oo54/pcubillos/Issues/2015-06-20_16-09-45.png (http://s361.photobucket.com/user/pcubillos/media/Issues/2015-06-20_16-09-45.png.html)

I did as you recommended on all four(4) computers... It is still encoding and no crashes. Will let you know when it finishes, but so far it is great!!!

mini-moose

21st June 2015, 09:37

10 min chunk is long enough to be safe. During that period you will have mix of static scenes (less bitrate needed) and dynamic (more bitrate needed).

I have to cautiously disagree. An average movie has anywhere between 4-8mins static end credits, and with this method the last segment gets as much bitrate as the rest of the movie. Further more, you would sometimes end up splicing a dynamic scene right in the middle, which would ruin the look ahead bitrate distribution.

Why not just switch to 1-pass ABR?, it would still be size sensitive and would save doing the first pass.

Atak_Snajpera

21st June 2015, 11:00

Viper714

21st June 2015, 15:24

Atak,

SUCCESS!!!

That actually did it!!! I had not checked the network cables yet as I wanted to try this last suggestion. Man these are beautiful numbers again!!

Atak,
Just a followup message. The encoding went perfectly. No issues. Not sure what was the reason for this but definitely using the x264 folder from version 1.18.1 was the solution. Thank you for your patience and advice in fixing this issue.

On another note have you ever given any thought to using the GPU along with the CPU or in lieu of the CPU? Are GPU's better than CPU's? I have seen some programs using them.

In either case, thank you again for your help. You have a awesome program for ripping and re-encoding movies.

:thanks: http://i361.photobucket.com/albums/oo54/pcubillos/Smileys/Small/SugarwareZ-003.gif (http://s361.photobucket.com/user/pcubillos/media/Smileys/Small/SugarwareZ-003.gif.html)

Viper714

21st June 2015, 21:11

if user selects 2pass mode in gui then app should accept that. 2pass has more information about complexity of next frames than blind abr. Besides abr is less accurate when you aim for specific size.

Personally I stoped using 2pass mode because I do not store my movies on optical discs anymore. My poor dvdrw has been collecting dust for few years.

Atak, I am curious as to your response. As you know I am re-encoding my Blu-Rays after I rip them with AnyDVD HD. Should I not be using 2 PASS when encoding them. Is their a better more efficient option while maintaining quality?

mini-moose

22nd June 2015, 09:06

Should I not be using 2 PASS when encoding them. Is their a better more efficient option while maintaining quality?

CRF would give you a better result unless you care about media sizes.

george84

22nd June 2015, 19:54

Could you please clarify the use of 2 audio channels.
I get problems when encoding an avs. I usually did the following:

1. Add New job and select an avs file.
2. Ripbot displays "gathering information" and then fills in video and audio.
3. In audio only "1" is green.
4. Start encoding
5. Encoding is done, but in the multiplex phase a second audio file "Encoded_Audio_1.aac" is referenced which doesn't exist, and multiplex fails.

When clicking on "2" in audio I notice that there is also a reference to avs file, but no profile is filled in.

How can I get rid of this reference to second audio file?

I just realised that after selecting a dummy file for the second audio file, I get the option "No Audio" in dropdown box for audio.
So this is a bug. This dropdown box should offer "No Audio" immediatly.

Atak_Snajpera

22nd June 2015, 20:44

Yes It looks like a bug. I will investigate this case tomorrow.

slalom

22nd June 2015, 21:58

How can I get rid of this reference to second audio file?
Before starting the job, check if audio 2 is "no audio"

george84

23rd June 2015, 08:19

Before starting the job, check if audio 2 is "no audio"

Yes, but I can only set it to "no audio" after first having selected a file (which will cause ripbot to gather information and take some time). So there is a workaround.

Question: Is there also possibility to have only audio 2? Should there be a option "No Audio" for audio 1?

Atak_Snajpera

23rd June 2015, 11:31

The reason behind this is, because the main master node is running on an SSD C:\ where files are being copied to and all this job creation/copy/deletion process is slowly chirping away my SSD's lifespan for each encode.

SSDs can easily write hundreds of TiB before they die. The best can even go beyond 1 PiB! http://techreport.com/review/27436/the-ssd-endurance-experiment-two-freaking-petabytes

Besides in 5 years we will be laughing at SATA III drives.

slalom

23rd June 2015, 14:49

Why don't you change the Temp directory to a non-ssd?
You will be equally as slow as what you suggest

Atak_Snajpera

23rd June 2015, 15:05

I like current portable solution where EncodingServer.exe doesn't need any additional local files to work. Nothing will be done in this matter. If you are so worried about your SSD then switch to HDD. I'm not going to waste my free time on this useless feature.

Viper714

23rd June 2015, 16:39

Why don't you change the Temp directoryto a non-ssd?
You will be equally as slow as what you suggest

I agree with Slalom. I have an SSD and had concern on the life of it. That is why I configured the .ini file to use a mechanical drive for my temp solution. It is very simple to do and a option to mitigate the SSD life span concern. Yes SSD's are quicker but more expensive. That is the toss up that each individual will have to deal with. I have over 400 movies in my library (two big boxes of Blu-ray disks safety stored) and I've used my mechanical drive for conversion.

Considering that RipBot is a free program there has to be a methodology where you look at features that are "Nice to Have" and then "Need to Have". The latter of course takes precedence.

george84

23rd June 2015, 16:57

Problem:

In selection of audio file the extension .avs is not among the supported audio formats. However when manually entering the name of a file with extension avs everything works.

It would be nice to have .avs extension supported, so that you can pick your audio avs file in explorer selection. See attachment

Atak_Snajpera

26th June 2015, 15:59

v1.18.3 final is out

Added: Bitrate Distribution Optimization for 2-pass mode in Distributed Encoding Mode (chunks with smaller complexity get less bitrate and vice versa)
Added: EncodingClient can shutdown servers when all jobs are finished (See EncodingClient.ini for more details)
Added: Shut down method has been also added for main executable (See entry ShutDownMethod=shutdown in RipBot264.ini)
Added: Autocrop in Batch mode
Added: Chunk Size for 2-pass mode is now adjustable.
Added: EncodingClient can alternatively create one chunk per node with equal number of frames to process (2-pass mode only)
Changed: Thanks to new BDO default chunk size for 2-pass mode could be reduced to 60s. (smaller chunk size = better work distribution)
Fixed: Crash in EncodingClient.exe
Fixed: DirectShowSource timeout error on some machines during encoding audio phase
Fixed: "NO AUDIO" option was missing if user selected .avs as video file
Updated: EncodingClient v1.6.0, EncodinServer v1.5.1, MKVToolnix v8.0.0, x265 1.7.243

soneca

26th June 2015, 16:25

Thank you! :cool:

jandor

26th June 2015, 23:05

Thanks! Atak

sneaker_ger

26th June 2015, 23:08

Added: Bitrate Distribution Optimization for 2-pass mode in Distributed Encoding Mode (chunks with smaller complexity get less bitrate and vice versa)
How does this work? Do you first do all first passes, then analyze the stats files and then adjust 2nd pass target bitrates accordingly?

mini-moose

26th June 2015, 23:26

How does this work? Do you first do all first passes, then analyze the stats files and then adjust 2nd pass target bitrates accordingly?

From what I can see it takes the bitrate achieved for each segment in first pass and uses it as the --bitrate respectively in 2nd pass. e.g. say chunk 1 ended up with 3500k in first pass, it will be then encoded with --bitrate 3500 in 2nd pass.

sneaker_ger

26th June 2015, 23:33

Your explanation does not really make sense to me. There has to be some kind of calculation to make overall average bitrate hit the target.

mini-moose

26th June 2015, 23:37

So it uses CRF for first pass?

Seems I was wrong. I thought it still uses a pre-defined bitrate on first pass but I don't see any such setting. I guess it just runs the first pass without specific bounds and then calculates the results relatively to the target bitrate.
I'm not 100% sure.

soneca

26th June 2015, 23:42

In a first attempt stopped at 100%, I tried again and it worked ... weird.

http://s20.postimg.org/cg6kkn1ql/RIPBOT1.png

Atak_Snajpera

27th June 2015, 11:41

How does this work? Do you first do all first passes, then analyze the stats files and then adjust 2nd pass target bitrates accordingly?

1) First pass is run in CQ mode with default CRF value instead of target bitrate
2) In CQ mode encoder adjusts bitrate on-fly according to current complexity of frames so at the end of encoding process I know which chunks should get less or more bitrate in second pass.

http://i.cubeupload.com/bKpxeS.png

Calculation is very simple. First I calculate average bitrate for all chunks. In this example average bitrate for first pass with default CRF value was 1438 kbps.
Next average bitrate from each chunk is divided by aforementioned 1438 kbps . All values are stored in BDO.2pass file.

BDO.2pass file
1438.1 <- average bitrate from first CQ mode pass (information only. Does not serve any purpose in 2-pass)
1.13343995549684 <- Chunk 1 bitrate will be multiplied by this value
0.49857450803143 <- Chunk 2 bitrate will be multiplied by this ...
1.4275780543773 <- Chunk 3 ....
1.07433419094639
0.537514776441138
0.709269174605382
1.25721438008483
0.794103330783673
0.719004241707809
1.84896738752521

3) In second pass target bitrate is simply multiplied by those values.

So If my target bitrate was 1000 kbps then first chunk gets 1133 kbps. Second 498 kbps. Third 1427 kbps and so on.

http://i.cubeupload.com/pJ9Okh.png

sneaker_ger

27th June 2015, 11:47

I see. Thank you for explaining.

slalom

27th June 2015, 22:31

What does this mean "ChunkSizeFor2passMode=60"?
Can I use less chunks and bigger number of frames?

Atak_Snajpera

28th June 2015, 10:34

yes you can but it does not make sense now. Smaller chunk is better for work distribution (Less server idle time). Also you lose less time if you decide to restart encoding after abort.

mini-moose

28th June 2015, 14:47

1) First pass is run in CQ mode with default CRF value instead of target bitrate
2) In CQ mode encoder adjusts bitrate on-fly according to current complexity of frames so at the end of encoding process I know which chunks should get less or more bitrate in second pass.

It seems like maybe a better way to do it than the old 2-pass method. But the quality still suffers greatly.

I encoded a blu-ray clip with regular 2-pass and with ripbot new distributed 2-pass method.
Everything other than the 2-pass method is identical: preset slow, --bitrate 3000, all x264 settings the same:

http://screenshotcomparison.com/comparison/132840