RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY [Archive] - Page 336

pepeq

25th February 2019, 08:37

I can confirm that I'm seeing the same behavior on the latest version making distributed encoding not possible. Swapped back to the version from end of 2018, and everything works fine. Hopefully this gets seen/addressed as the distributed encoding is the most enticing feature of RipBot264 for me.

I see the same problem in my DE-environment, too.

Atak_Snajpera

25th February 2019, 13:31

On the latest version 1.24.0 with the latest updates. So most likely ffms2 is still based on the old ffmpeg version. Good to know once that is updated I won't need to do lossless transcoding any longer for VC-1 sources.

Is latest ffms2 frame accurate?

Ps. VC1 has been always pain in the ass... I just hope that stupid codec will just die!

defalopii

25th February 2019, 14:03

Hello, ive been using ripbot since 2012, i just want to ask to its developer, atak, are You planning to support encoding using vp9 in the near future of this great app? As we know vp9 comparable to x265 which ripbot already support for it long time ago, but x265 is not compatible with html5 video streaming, vp9 is, so i ask this question

Atak_Snajpera

25th February 2019, 14:25

Ryushin

25th February 2019, 15:41

Is latest ffms2 frame accurate?

Ps. VC1 has been always pain in the ass... I just hope that stupid codec will just die!

I don't know if it is frame accurate or not. Just responding to sneaker_ger.

I have two VC-1 sources (Bladerunner and Terminator 2) that I'm waiting to encode so I can do tests on those. I'll download the latest ffms2 2.23.1 and let you know.

AtaK: Do I need 32bit or 64bit? I thought you moved to 64 bit a couple of years ago.

defalopii

25th February 2019, 15:42

Most likely h.266/VVC or AV1. VP9 will die soon.

so x265 will die soon also? today is streaming age, encoding x265 with powerful DE in this app is a little less benefit if it cant be streamed online. Or are u planning to support encoding AV1 in the near future?

Atak_Snajpera

25th February 2019, 15:58

so x265 will die soon also? today is streaming age, encoding x265 with powerful DE in this app is a little less benefit if it cant be streamed online. Or are u planning to support encoding AV1 in the near future?

VP9 will simply be replaced by AV1 in the same way how Vorbis was replaced by OPUS. Problem with AV1 today is that implementation is extremely sloooow and quality difference (vs x265) is also not amazing.

Atak_Snajpera

25th February 2019, 16:00

I don't know if it is frame accurate or not. Just responding to sneaker_ger.

I have two VC-1 sources (Bladerunner and Terminator 2) that I'm waiting to encode so I can do tests on those. I'll download the latest ffms2 2.23.1 and let you know.

AtaK: Do I need 32bit or 64bit? I thought you moved to 64 bit a couple of years ago.
So what does this mean?
https://forum.doom9.org/showthread.php?p=1864099#post1864099

Ryushin

25th February 2019, 16:17

So what does this mean?
https://forum.doom9.org/showthread.php?p=1864099#post1864099

Was that the latest version? If so, I guess it is not frame accurate.

Perhaps the only way to solve this is to to do the intermediate step of converting VC-1 to h.264 lossless. Then you get get rid of all the workarounds for handling VC-1.

Atak_Snajpera

25th February 2019, 16:35

Was that the latest version? If so, I guess it is not frame accurate.

Perhaps the only way to solve this is to to do the intermediate step of converting VC-1 to h.264 lossless. Then you get get rid of all the workarounds for handling VC-1.

That intermediate step would only kill any time savings in DE mode... In that case I would just use regular mode on the fastest machine in the house.

This looks like the latest version. Check and let me know how it went
https://forum.doom9.org/showthread.php?p=1866411#post1866411

Just copy ffms2.dll and ffmsindex.exe to ..\Tools\AviSynth plugins\ffms\x64 folder

...and start with fresh job because index file may not be compatible with older version.

UPDATE: Latest version is still broken because it sees all frames as keyframes. However My initial test shows that encoded VC1 file is seamless (no frame corruption or skipped frames where old chunks ends and new one starts)
# keyframe format v1
fps 0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

Ryushin

25th February 2019, 17:00

That intermediate step would only kill any time savings in DE mode... In that case I would just use regular mode on the fastest machine in the house.

The lossless conversion really did not take very long. Takes a lot longer to suck in a 4K then for it to do a lossless conversion. The advantage is that DE is still running why the VC-1 source is being demuxed and converted to lossless.

I suppose I can use that one fast machine in the house to do the VC-1s or I can just convert it to lossless myself for those sources. Once I convert video.mkv, can I delete the video.mkv.ffindex* files and will the index be built automatically when the job starts?

Atak_Snajpera

25th February 2019, 17:07

The lossless conversion really did not take very long. Takes a lot longer to suck in a 4K then for it to do a lossless conversion. The advantage is that DE is still running why the VC-1 source is being demuxed and converted to lossless.

I suppose I can use that one fast machine in the house to do the VC-1s or I can just convert it to lossless myself for those sources. Once I convert video.mkv, can I delete the video.mkv.ffindex* files and will the index be built automatically when the job starts?

If you remove video.mkv.ffindex then Encodingclient will automatically reindex video.mkv.

I've updated my post
https://forum.doom9.org/showthread.php?p=1866770#post1866770

katodevin

25th February 2019, 17:22

I see the same problem in my DE-environment, too.

Atak_Snajpera- just hoping you saw that some of us were having issues with distributed encoding on the latest release. Has to do with the files within the script being referenced in a local manner rather than in the shared format.

See post #16742 for details.

Ryushin

25th February 2019, 17:23

If you remove video.mkv.ffindex then Encodingclient will automatically reindex video.mkv.

I've updated my post
https://forum.doom9.org/showthread.php?p=1866770#post1866770

I'll try that new version of ffms. I assume I can just delete the ffindex files. I'll try that one job, and if it works, I'll nuke the ffindex files for the 60 or so jobs I have in the cue.

Atak_Snajpera

25th February 2019, 17:34

I'll try that new version of ffms. I assume I can just delete the ffindex files. I'll try that one job, and if it works, I'll nuke the ffindex files for the 60 or so jobs I have in the cue.

For testing purposes use some low resolution (720p) to speed up whole process. I just need to know that everything is correctly encoded at stitches.

Ryushin

25th February 2019, 18:24

For testing purposes use some low resolution (720p) to speed up whole process. I just need to know that everything is correctly encoded at stitches.

I probably won't be able to get back with you about this for a few hours. Have to take care of something else right now.

Atak_Snajpera

25th February 2019, 18:32

I probably won't be able to get back with you about this for a few hours. Have to take care of something else right now.

No problem. My current testing shows that latest version of ffms2 is NOT frame accurate for AVC! (new chunk does not start from correct frame) VC1 seems to be ok despite detecting all frames as I-frame.

Atak_Snajpera

25th February 2019, 19:49

Latest ffms2 also is not frame accurate for HEVC! (some frames are missing at joining point).

https://i.imgsafe.org/43/43863b318f.png

https://i.imgsafe.org/43/4386833ae6.png

Ryushin

25th February 2019, 21:38

The VC-1 source I used tested out great. Too bad it's broken for everything else.

VC-1 is just a pain. Back to making h.264 lossless before the conversion for me.

Atak_Snajpera

26th February 2019, 13:38

The VC-1 source I used tested out great. Too bad it's broken for everything else.

VC-1 is just a pain. Back to making h.264 lossless before the conversion for me.

Do not worry. I will be using different version for different codecs.
ffms2 2014 -> AVC
ffms2 2017 -> HEVC
ffms2 2019 -> VC1 and rest

Ryushin

26th February 2019, 14:42

Do not worry. I will be using different version for different codecs.
ffms2 2014 -> AVC
ffms2 2017 -> HEVC
ffms2 2019 -> VC1 and rest

Let me know when this is implemented and I will start adding the vc1 jobs.

byteshare

26th February 2019, 17:15

so x265 will die soon also? today is streaming age, encoding x265 with powerful DE in this app is a little less benefit if it cant be streamed online
x265 is already used to stream. Netflix uses it if your device supports it.

Most likely h.266/VVC or AV1. VP9 will die soon.
Is H.266 supposed to come out soon?! It seems 264 still hasn't completely been replaced by 265 yet.

pepeq

26th February 2019, 17:25

Atak_Snajpera- just hoping you saw that some of us were having issues with distributed encoding on the latest release. Has to do with the files within the script being referenced in a local manner rather than in the shared format.

See post #16742 for details.

The workaround described in post #16742 works, but remember to create a network connection to your movie-source, which is valid from an elevated shell also.
If you create a network connection (or do a 'net use') with your ordinary user-rights, the workaround does not work.
Reason: RipBot264 runs in Administrator-mode and therefore the movie-source has to be accessable with elevated (Administrator-) rights.

Hope that helps.

Atak_Snajpera

26th February 2019, 17:36

Let me know when this is implemented and I will start adding the vc1 jobs.

Now !

slalom

26th February 2019, 21:25

A problem with version 1.24.1
Although starting and stopping on DE was fixed, if you abort a job and try to restart it, the problem re-appears

slalom

26th February 2019, 22:03

I restarted all PCs
Only one server working in DE (the one with the jobs). All the others are starting and stopping

Ryushin

26th February 2019, 22:45

Now !

Thank you Atak. You are the greatest.

Ryushin

27th February 2019, 03:36

Afraid I'm in the same boat now. None of my DE jobs are running now. Restored from backup, turned off auto updates, and everything is working again.

pepeq

27th February 2019, 09:07

Afraid I'm in the same boat now. None of my DE jobs are running now. Restored from backup, turned off auto updates, and everything is working again.

see post #16773, this may help until Atak has fixed it.

Atak_Snajpera

27th February 2019, 12:09

pepeq

27th February 2019, 12:51

Use latest version only if you have no old jobs in queue. Old jobs may not start due to incompatibility with ffms2.dll and old .index file.

You may try to manually fix this by editing each jobx.avs file.
If your source video is not AVC or VC1 then change ffms_latest to 2017 in line below #VideoSource

#VideoSource
LoadPlugin("C:\Users\Dave\Documents\Delphi_Projects\RipBot264\_Compiled\Tools\AviSynth plugins\ffms\ffms_latest\x64\ffms2.dll")

#VideoSource
LoadPlugin("C:\Users\Dave\Documents\Delphi_Projects\RipBot264\_Compiled\Tools\AviSynth plugins\ffms\2017\x64\ffms2.dll")

notepad++ should be able to do that in batch.

@Atak: but you know, that this does not solve the problem described in post #16773.

Ryushin

27th February 2019, 15:04

Use latest version only if you have no old jobs in queue. Old jobs may not start due to incompatibility with ffms2.dll and old .index file.

You may try to manually fix this by editing each jobx.avs file.
If your source video is not AVC or VC1 then change ffms_latest to 2017 in line below #VideoSource

#VideoSource
LoadPlugin("C:\Users\Dave\Documents\Delphi_Projects\RipBot264\_Compiled\Tools\AviSynth plugins\ffms\ffms_latest\x64\ffms2.dll")

#VideoSource
LoadPlugin("C:\Users\Dave\Documents\Delphi_Projects\RipBot264\_Compiled\Tools\AviSynth plugins\ffms\2017\x64\ffms2.dll")

notepad++ should be able to do that in batch.

I had not idea that notepad++ would do that. I will need to look into that.

If anyone has the Linux subsystem installed, this command will do it as well (one line):
find /mnt/d/Temp/RipBot264temp/ -maxdepth 2 -type f -regex '.*job[0-9][0-9][0-9][0-9].avs?' -exec sed -i 's/ffms\\ffms_latest\\x64/ffms\\2017\\x64/g'
{} \;

Ryushin

27th February 2019, 15:33

My jobs are stuck right now. Just says starting... but never does anything.

DE Server shows:
Encoding started...
""\\SONNY\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\SONNY\RipBot264temp\job1107\Chunks\1.avs" -strict -1 -f yuv4mpegpipe - | "\\SONNY\Ripbot264temp\tools\x265\x265_x64.exe" --seek 0 --colorprim bt709 --transfer bt709 --colormatrix bt709 --crf 18 --fps 24000/1001 --min-keyint 24 --keyint 240 --frames 1434 --sar 1:1 --profile main10 --output-depth 10 --sar 1:1 --ctu 32 --y4m --pools "+" --output "\\SONNY\RipBot264temp\job1107\Chunks\1.265" -"

I've made the changes to the job*.avs files for the change for ffms. Even deleted the index files. Not sure what is going on yet.

Question: Is there an option to not build the index after demuxing the job but rather let it be created when the job starts? It would make pulling in jobs faster for me.

Edit: Odd, after 10 minutes, the jobs actually started. <que twilight zone music>

Ryushin

27th February 2019, 15:43

Atak, I removed the couple of VC-1 jobs I had. Do you recommend deleting all the indexes for all the jobs or they should they be fine once the job*.avs script is updated?

Atak_Snajpera

27th February 2019, 15:56

Atak, I removed the couple of VC-1 jobs I had. Do you recommend deleting all the indexes for all the jobs or they should they be fine once the job*.avs script is updated?

They should be fine.

katodevin

27th February 2019, 19:32

@Atak: but you know, that this does not solve the problem described in post #16773.

Seconded. I have no jobs in queue, created jobs with the latest version, and cannot get distributed encoding to start on any of the remote computers.

Swapping back to the old version works fine.

slalom

27th February 2019, 19:47

egres

28th February 2019, 17:06

Seconded. I have no jobs in queue, created jobs with the latest version, and cannot get distributed encoding to start on any of the remote computers.

Swapping back to the old version works fine.

Reported the same issue a couple days ago, #16712 . I just tried again today. When I fired up RP, I got 3 updates: core.zip,ffms.zip and MediaInfo.zip. I then created a new job. Same result as you. Even the local encoder won't start

P.S: If I do a fresh install from RipBot264v1.24.0.7z and making sure updates are disabled, everything works.

slalom

28th February 2019, 22:08

Seconded. I have no jobs in queue, created jobs with the latest version, and cannot get distributed encoding to start on any of the remote computers.
Did that too
Fresh install everywhere, one job added.
Nothing starts

pepeq

1st March 2019, 13:24

Did that too
Fresh install everywhere, one job added.
Nothing starts

For all of you, who are having the error, that RB does not start on the DE-server machines, but only on the DE-server itself:

An easier work-around is to reference the source video-file via a network-path on the DE-server, when you ADD or EDIT the job.

Example:
In menu Open when adding or editing a RB-job, do not select the source-video via a local disk-path (e.g. D:\videos\source.mkv), but via the UNC-network path (e.g. \\mymachine\videos\source.mkv).

The only requirement is to create a network-share (\\mymachine\videos) to your source-video directory (D:\videos), which is accessible from your DE-client-machines. Then you do not to have to create a network-mapping on all your DE-clients.

If you do so, then in all files jobX/chunks/X.avs the local path D:\videos\... is substituted with the network-path \\mymachine\videos\... and your DE-clients can access the source-video.

Hope that helps.

slalom

1st March 2019, 15:50

Ok, I did that
Now all servers say "starting" but nothing happens, cpu is idle

pepeq

1st March 2019, 16:01

Ok, I did that
Now all servers say "starting" but nothing happens, cpu is idle

This is another problem.
See #16556, maybe this will help you.
It helped a few other users (see #16560 and #16745)

byteshare

1st March 2019, 17:13

This is another problem.
See #16556, maybe this will help you.
It helped a few other users (see #16560 and #16745)
I don't have those issues but if I click abort and then start a job after the encodes never start but keep trying, but if I click abort again > close and then open RipBot > Click encode it works again.

slalom

1st March 2019, 18:06

This is another problem.
See #16556, maybe this will help you.
It helped a few other users (see #16560 and #16745)
I ran Resource Monitor
Didn't find anything strange
I don't have those issues but if I click abort and then start a job after the encodes never start but keep trying, but if I click abort again > close and then open RipBot > Click encode it works again.
Seems to work. Do you have to do that once for all jobs?

All this is not normal behaviour
An easier work-around is to reference the source video-file via a network-path on the DE-server, when you ADD or EDIT the job.

Example:
In menu Open when adding or editing a RB-job, do not select the source-video via a local disk-path (e.g. D:\videos\source.mkv), but via the UNC-network path (e.g. \\mymachine\videos\source.mkv).

The only requirement is to create a network-share (\\mymachine\videos) to your source-video directory (D:\videos), which is accessible from your DE-client-machines. Then you do not to have to create a network-mapping on all your DE-clients.

If you do so, then in all files jobX/chunks/X.avs the local path D:\videos\... is substituted with the network-path \\mymachine\videos\... and your DE-clients can access the source-video.

Hope that helps.
Did anyone notice network overload?
The speed is of course 1Gbps

slalom

2nd March 2019, 12:07

The problem with me is Network I/O waits
When this happens and the encoding doesn't start, there is high memory and network usage

Atak_Snajpera

2nd March 2019, 14:17

The problem with me is Network I/O waits
When this happens and the encoding doesn't start, there is high memory and network usage

And what does Process Hacker show regarding cpu usage by ffmpeg.exe ?

Something tells me that due to incompatible .index file ffms2 plugin simply re-indexes you video file on fly and hence high network usage. (each server does the same simultaneously)

slalom

2nd March 2019, 18:14

Not much

https://i.ibb.co/vzwXgqC/image.jpg (https://ibb.co/f01DBFV)

and resource monitor

https://i.ibb.co/gtfFnCb/2.jpg (https://ibb.co/44XgQxw)

nekrosoft13

3rd March 2019, 05:45

I managed to get my DE servers to work again by mounting a share on each of them so that the path that is set on the avs file is the same locally for each of them.

Basically, in the avs file, on each DE server,
FFVideoSource: Failed to open 'Y:\filename.mkv' (\\PCNAME\RipBot264temp\job4\Chunks\6.avs, line 7)
the "Y:" is wrong, it should be \\PCNAME\(...)". The path that is set in the avs file is only valid in the machine running ripbot.

My fix was making that path also valid on each DE server by sharing and mapping the share so that it matches.

This only happens since the last version, this was working fine previously, but the problem is definitly that.

fK

same issue, updated today, and it broke same day

Ryushin

3rd March 2019, 15:44

So I'm still encountering this issue.

I rip a 4K disc to the hard drive that has seamless branching, such as Incredibles 2. I then use Ripbot to pull in the movie and process it. The output will have out of sync audio towards the end of the movie. I think this only happens on 4K discs that are towards 2 hours or longer.

The work around I've been doing is to use MakeMKV to create a video file from the same source and then pull that into Ripbot to process it and it comes out fine. Only caveat seems to be Ripbot throws and error when pulling in a mkv file that has a TrueHD stream in it.
Decoding Error
FFAudioSource: No audio track found
(D:\Temp\Ripbot264temp\Job1023\getinfo.avs, line 4)

So I create two MKVs, one with AC3 which Ripbot pulls in fine and is processed, and the other with TrueHD, that I mux in the TrueHD audio stream after the Ripbot finishes with the first file.

Still have this issue 4K and seamless branching. Just had to go through this with Ralph Breaks the Internet and Snow White and the Huntsman Extended edition. RB still throws and error when pulling in a mkv that has a TrueHD stream in it. Though I suspect that is caused by it being the first audio stream? Next time I'll remux it and make it the second audio stream to see if the error still occurs.

pepeq

4th March 2019, 14:45

For all of you, who are having the error, that RB does not start on the DE-server machines, but only on the DE-server itself:

An easier work-around is to reference the source video-file via a network-path on the DE-server, when you ADD or EDIT the job.

Example:
In menu Open when adding or editing a RB-job, do not select the source-video via a local disk-path (e.g. D:\videos\source.mkv), but via the UNC-network path (e.g. \\mymachine\videos\source.mkv).

The only requirement is to create a network-share (\\mymachine\videos) to your source-video directory (D:\videos), which is accessible from your DE-client-machines. Then you do not to have to create a network-mapping on all your DE-clients.

If you do so, then in all files jobX/chunks/X.avs the local path D:\videos\... is substituted with the network-path \\mymachine\videos\... and your DE-clients can access the source-video.

Hope that helps.

Problem solved with latest update RB 1.24.1, the files chunks/X.avs are correct now.
Thanks @Atak!