Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 31st July 2019, 20:19   #17061  |  Link
gabbett1
Registered User
 
Join Date: Sep 2015
Posts: 46
Quote:
Originally Posted by Atak_Snajpera View Post
Screenshots ,please.
LOL, figures. It must have updated or something. It's working fine. disregard. Thanks.
gabbett1 is offline   Reply With Quote
Old 1st August 2019, 17:44   #17062  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
v1.25.0
Code:
Added: Variable priority mode to EncodingServer. 
       In this mode EncodingClient will be automatically changing encoding priority in order to balance progress on multiple servers running on the same machine.
Added: EncodingClient now also shows encoding priority set on server side and selected encoding numa node.
Fixed: EncodingClient is now copying file to shared folder in buffered mode. 
       Windows disables buffered write if write buffer in application is higher than 16MiB. 
       Because of this write buffer has been reduced from 64MiB to 16MiB. 
       Buffered write significantly reduces copy time on machine with huge amount of RAM. 
Changed: lsmash is now a default decoder for all formats except VC1. FFMS will be still used for VC1.
Fixed: EncodingServer not starting on some Xeon machines with high number of threads.
Fixed: few minor bugs found since last version.
Progress balancing OFF


Progress balancing ON


Important message
Use variable priority only if:
1) you encode on single machine (for example with high number of cores/threads)
2) ...and use multiple servers to get full cpu usage
3) ...and chunks size is set to 0 (one big chunk for each active server)

Otherwise encoding time will be bigger than in default mode.

Last edited by Atak_Snajpera; 1st August 2019 at 18:51.
Atak_Snajpera is offline   Reply With Quote
Old 2nd August 2019, 17:50   #17063  |  Link
byteshare
ByteShare
 
byteshare's Avatar
 
Join Date: Sep 2014
Location: On the Internet
Posts: 560
Quote:
Originally Posted by Atak_Snajpera View Post
v1.25.0
Code:
Added: Variable priority mode to EncodingServer. 
       In this mode EncodingClient will be automatically changing encoding priority in order to balance progress on multiple servers running on the same machine.
Added: EncodingClient now also shows encoding priority set on server side and selected encoding numa node.
Fixed: EncodingClient is now copying file to shared folder in buffered mode. 
       Windows disables buffered write if write buffer in application is higher than 16MiB. 
       Because of this write buffer has been reduced from 64MiB to 16MiB. 
       Buffered write significantly reduces copy time on machine with huge amount of RAM. 
Changed: lsmash is now a default decoder for all formats except VC1. FFMS will be still used for VC1.
Fixed: EncodingServer not starting on some Xeon machines with high number of threads.
Fixed: few minor bugs found since last version.
Awesome! Thank you.
I'm seeing *.lwi files now, is that from the lsmash?

Last edited by byteshare; 3rd August 2019 at 04:33.
byteshare is offline   Reply With Quote
Old 3rd August 2019, 06:12   #17064  |  Link
byteshare
ByteShare
 
byteshare's Avatar
 
Join Date: Sep 2014
Location: On the Internet
Posts: 560
x265 Unable to open input file

I'm using this custom script in the custom area:
Code:
video=video.ConvertBits(16).ConvertToStacked().SMDegrain(tr=3,thSAD=400,thSADC=150,refinemotion=false,contrasharp=true,plane=4,pel=2,prefilter=4,Truemotion=False,chroma=true,hpad=32,vpad=32,str=2,amp=1,lsb_in=true,lsb=true,lsb_out=true).ConvertFromStacked().ConvertBits(8).FineDehalo().FastLineDarkenMOD4().LSFmod()
The AVISynth Preview doesn't throw an error and it also works fine in AVSpmod.
On the Encoding server I either see it get to encoding of x265 and then crash (see below):
Code:
1|153422453|ENCODING_STARTED
2|153422750|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=13;RAM=40;DECODER=0;ENCODER=0;OTHER=13;ENCODING_PRIORITY=low;
3|153423078|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=13;RAM=40;DECODER=0;ENCODER=0;OTHER=13;ENCODING_PRIORITY=low;
4|153423421|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=13;RAM=40;DECODER=0;ENCODER=0;OTHER=13;ENCODING_PRIORITY=low;
5|153423890|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=18;RAM=40;DECODER=2;ENCODER=0;OTHER=16;ENCODING_PRIORITY=low;
6|153424265|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=18;RAM=40;DECODER=2;ENCODER=0;OTHER=16;ENCODING_PRIORITY=low;
7|153424625|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=18;RAM=40;DECODER=2;ENCODER=0;OTHER=16;ENCODING_PRIORITY=low;
8|153425125|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=28;RAM=40;DECODER=6;ENCODER=0;OTHER=22;ENCODING_PRIORITY=low;
9|153425500|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=28;RAM=40;DECODER=6;ENCODER=0;OTHER=22;ENCODING_PRIORITY=low;
10|153425859|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=28;RAM=40;DECODER=6;ENCODER=0;OTHER=22;ENCODING_PRIORITY=low;
11|153426359|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=40;DECODER=6;ENCODER=0;OTHER=21;ENCODING_PRIORITY=low;
12|153426734|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=40;DECODER=6;ENCODER=0;OTHER=21;ENCODING_PRIORITY=low;
13|153427093|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=40;DECODER=6;ENCODER=0;OTHER=21;ENCODING_PRIORITY=low;
14|153427609|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=22;RAM=42;DECODER=3;ENCODER=0;OTHER=19;ENCODING_PRIORITY=low;
15|153427984|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=22;RAM=42;DECODER=3;ENCODER=0;OTHER=19;ENCODING_PRIORITY=low;
16|153428375|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=22;RAM=42;DECODER=3;ENCODER=0;OTHER=19;ENCODING_PRIORITY=low;
17|153428921|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=46;DECODER=4;ENCODER=0;OTHER=23;ENCODING_PRIORITY=low;
18|153429328|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=46;DECODER=4;ENCODER=0;OTHER=23;ENCODING_PRIORITY=low;
19|153429765|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=27;RAM=46;DECODER=4;ENCODER=0;OTHER=23;ENCODING_PRIORITY=low;
20|153430390|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ;CPU=72;RAM=47;DECODER=16;ENCODER=0;OTHER=56;ENCODING_PRIORITY=low;
21|153430718|ENCODING_PROGRESS=1 192.168.1.90:1000 -> ERROR;CPU=72;RAM=47;DECODER=16;ENCODER=0;OTHER=56;ENCODING_PRIORITY=low;
22|153430984|SERVER_IDLE
23|153431500|ENCODING_STARTED
Or I see an error in the main (not TCP Communication):
Code:
x265 [error]: unable to open input file <->
Even if I close the Encoding Server and restart that and no matter how many times the Encoding server retries a chunk it doesn't seem to work but if I close RipBot and try again it works?

Like I said it works but wasn't sure if there was something more "interesting" going on.
byteshare is offline   Reply With Quote
Old 3rd August 2019, 20:28   #17065  |  Link
BLKMGK
Registered User
 
Join Date: Feb 2008
Posts: 145
Quote:
Originally Posted by BLKMGK View Post
Any ideas on what is causing this (I'm using DE)?

Quote:
Encoding started...
""\\XXXXX\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\XXXXX\RipBot264temp\job883\Chunks\5.avs" -strict -1 -f yuv4mpegpipe - | "\\XXXXXXX\Ripbot264temp\tools\x265\x265_x64.exe" --seek 4 --colorprim bt709 --transfer bt709 --colormatrix bt709 --crf 18 --fps 24000/1001 --min-keyint 24 --keyint 240 --frames 1441 --sar 1:1 --preset slow --ctu 32 --y4m --pools "+" --output "\\XXXXXXX\RipBot264temp\job883\Chunks\5.265" -"
x265 [error]: unable to open input file <->

I have a number of Windows clients. For some reason TV shows NEVER show this error but movies often (but not always) do. It looks like a permissions issue but the share is wide open and if I type in the address on the client machine I get right in and appear to have Full Control as intended. I'm at wit's end and have been unable to find a correlation to understand why this doesn't happen on everything. I feel like it must be something silly but cannot find it and have been trying for at least a month.

This slow my encoding greatly to say the least when it occurs Help please!

P.S. When it occurs no client except the host is able to encode.
Still stuck on this! It appears to only occur on jobs for which I'm running AVISynth filtering. I'd really appreciate some input on what to look for as this filtering is really something I need, thanks!

Last edited by BLKMGK; 3rd August 2019 at 20:43.
BLKMGK is offline   Reply With Quote
Old 3rd August 2019, 22:29   #17066  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
Show us your avisynth script.
Atak_Snajpera is offline   Reply With Quote
Old 4th August 2019, 05:05   #17067  |  Link
BLKMGK
Registered User
 
Join Date: Feb 2008
Posts: 145
Quote:
Originally Posted by Atak_Snajpera View Post
Show us your avisynth script.
I'm clicking on the AVIsynth button, selecting the DeNoise dropd-own, and then selecting the bottom-most filter. I then either leave it at zero or bump the level to 2 depending upon the video source. I also allow it to autocrop.

Nothing custom. It acts as if there's a file not being shared and it cannot access it. If I do videos onesy twosey it doesn't seem to happen as often but if I stack up a queue of say 9 of them I start running into this. I have managed to process videos with these filters enabled but when this occurs it always seems to be on filtered videos.

The share is set to allow "everybody" full control and I can use the filemanager on the problem systems to get to the source files but running the CMD in the job folder results in the same error, I'm not sure what it cannot access. I have a stack of about 4 videos doing it right now but I also just added some things that are much shorter and lower resolution - they will process just fine I'm sure. I'm now on 1.25 on most of my machines but my Win7 cannot run that encoder Server for some reason and is on an older one. A brand new updated Win10 machine is having this issue too though.
BLKMGK is offline   Reply With Quote
Old 4th August 2019, 10:57   #17068  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
Quote:
Originally Posted by BLKMGK View Post
I'm clicking on the AVIsynth button, selecting the DeNoise dropd-own, and then selecting the bottom-most filter. I then either leave it at zero or bump the level to 2 depending upon the video source. I also allow it to autocrop.

Nothing custom. It acts as if there's a file not being shared and it cannot access it. If I do videos onesy twosey it doesn't seem to happen as often but if I stack up a queue of say 9 of them I start running into this. I have managed to process videos with these filters enabled but when this occurs it always seems to be on filtered videos.

The share is set to allow "everybody" full control and I can use the filemanager on the problem systems to get to the source files but running the CMD in the job folder results in the same error, I'm not sure what it cannot access. I have a stack of about 4 videos doing it right now but I also just added some things that are much shorter and lower resolution - they will process just fine I'm sure. I'm now on 1.25 on most of my machines but my Win7 cannot run that encoder Server for some reason and is on an older one. A brand new updated Win10 machine is having this issue too though.
It looks like you are using GPU filter KNLMeansCL for denoising. I suppose that filter is having some issues accessing your GPU. Perhaps device id is not set correctly. I'm just guessing now...

Run EncodingServer.exe on each machine (host as well) and take a screenshot.

Last edited by Atak_Snajpera; 4th August 2019 at 10:59.
Atak_Snajpera is offline   Reply With Quote
Old 4th August 2019, 17:29   #17069  |  Link
BLKMGK
Registered User
 
Join Date: Feb 2008
Posts: 145
Host both idle and encoding at 5ghz




Second machine - Ryzen 3900X and yes it's blowing away the other machine when it encodes! :-O





I hadn't realized this leveraged the GPU and it hadn't been set on the Ryzen machine or my others that I've not posted so as not to flood the site with images. I did set the GPU on the second box (it's crappy but usually headless) but same error currently. If the GPU is the issue, is there a CPU based filter that's as good? IMO this one removes film grain and some glitches amazingly well! It may be that my GPU in the second machine is not good enough perhaps? I've checked another machine with just an Intel GPU and no joy there either.

Please let me know if I can provide anything else to figure this one out, I really appreciate the assistance. I'd be happy to switch to a CPU based filter if I can maintain quality.

I'll try hosting an encode on my Ryzen box too to see if it encounters this problem but cannot do it this moment.
BLKMGK is offline   Reply With Quote
Old 4th August 2019, 18:01   #17070  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
If you are going to use KNLMeansCL in DE mode then you must:
1) have decent gpus in all PC's. GeForce 210 is just a crap GPU (Supports up to OpenCL 1.1 [knlmeansCL requires 1.2] and only two compute units!).
2) manually set device id in encoding server if you have more than one GPU. One of your machines has Intel GPU and GeForce 1070 for example.
This will tell encodingserver to use GeForce 1070 (faster)
EncodingServer.exe /knlmeanscl-opencl-device-id 1

In your case It might be a good idea to run one server using Intel GPU and other GeForce 1070 .
EncodingServer.exe /knlmeanscl-opencl-device-id 0
EncodingServer.exe /knlmeanscl-opencl-device-id 1

This way you will be using literally everything what can compute CPU+iGPU+dGPU

If you want you can also try running KNLMeansCL on CPU but be warned it will be veeeeeeeerrrrrrryyyyy slooooooooow.
EncodingServer.exe /knlmeanscl-opencl-device-type CPU

Last edited by Atak_Snajpera; 4th August 2019 at 18:13.
Atak_Snajpera is offline   Reply With Quote
Old 4th August 2019, 20:11   #17071  |  Link
BLKMGK
Registered User
 
Join Date: Feb 2008
Posts: 145
Quote:
Originally Posted by Atak_Snajpera View Post
If you are going to use KNLMeansCL in DE mode then you must:
1) have decent gpus in all PC's. GeForce 210 is just a crap GPU (Supports up to OpenCL 1.1 [knlmeansCL requires 1.2] and only two compute units!).
2) manually set device id in encoding server if you have more than one GPU. One of your machines has Intel GPU and GeForce 1070 for example.
This will tell encodingserver to use GeForce 1070 (faster)
EncodingServer.exe /knlmeanscl-opencl-device-id 1

In your case It might be a good idea to run one server using Intel GPU and other GeForce 1070 .
EncodingServer.exe /knlmeanscl-opencl-device-id 0
EncodingServer.exe /knlmeanscl-opencl-device-id 1

This way you will be using literally everything what can compute CPU+iGPU+dGPU

If you want you can also try running KNLMeansCL on CPU but be warned it will be veeeeeeeerrrrrrryyyyy slooooooooow.
EncodingServer.exe /knlmeanscl-opencl-device-type CPU
Thank you, that's a HUGE help! What's the minimum GPU you'd recommend? I assume a 1050 would work? I have one of those in a Linux machine for realtime streaming, the 210 was bought simply to get me video on the Ryzen box but I'll upgrade it. I wish I could interleave jobs and send the ones that don't require GPU to just those machines lol. The Ryzen box does require two encoding servers to get full utilization BTW, it's close to triple the speed of my 8700K I'll see how it handles the filter on the CPU

P.S. what are these *.lwi files now showing up with the new client? Seem to contain data about the various videos.
BLKMGK is offline   Reply With Quote
Old 4th August 2019, 20:19   #17072  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
*.lwi is an index file for video file. It is required to have frame accurate seeking.
Atak_Snajpera is offline   Reply With Quote
Old 4th August 2019, 22:01   #17073  |  Link
BLKMGK
Registered User
 
Join Date: Feb 2008
Posts: 145
Must require something other than OpenCL 1.2, I've got a GTX 680 that claims to support OpenCL 1.2 with a whopping 4 compute units that also hits this error - argh! Looks like I might need more than one new video card! :O
BLKMGK is offline   Reply With Quote
Old 5th August 2019, 10:35   #17074  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
Try with 1050
Atak_Snajpera is offline   Reply With Quote
Old 5th August 2019, 16:58   #17075  |  Link
byteshare
ByteShare
 
byteshare's Avatar
 
Join Date: Sep 2014
Location: On the Internet
Posts: 560
Quote:
Originally Posted by Atak_Snajpera View Post
*.lwi is an index file for video file. It is required to have frame accurate seeking.
Thank you for clarifying that.
byteshare is offline   Reply With Quote
Old 5th August 2019, 18:58   #17076  |  Link
userx
Registered User
 
userx's Avatar
 
Join Date: Mar 2016
Location: Austria
Posts: 28
assertion failed on 1st pass

Hello,
For some time the first pass in 2-pass mode won't start at my remote machine. 2nd pass works like expected. If I start a job on commandline manually I get following output:
Code:
PS Microsoft.PowerShell.Core\FileSystem::\\USERX-MASTER\RipBot264temp\job5\Chunks> .\39.cmd
"\\HOST\RipBot264temp\job5\Chunks"
CMD.EXE wurde mit dem oben angegebenen Pfad als aktuellem Verzeichnis gestartet.
UNC-Pfade werden nicht unterstützt.
Stattdessen wird das Windows-Verzeichnis als aktuelles Verzeichnis gesetzt.
""\\HOST\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\HOST\RipBot264temp\job5\Chunks\39.avs" -strict -1 -f yuv4mpegpipe - | "\\HOST\Ripbot264temp\tools\x264\x264_x64.exe" --seek 48 --colorprim bt709 --transfer bt709 --colormatrix bt709  --opencl --opencl-device 0 --opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead.clbin" --pass 1 --bitrate 3000 --stats "\\HOST\RipBot264temp\job5\Chunks\39.stats" --fps 25 --force-cfr  --min-keyint 25 --keyint 250 --frames 1488 --sar 1:1 --level 4.1 --aud --nal-hrd vbr --vbv-bufsize 25000 --vbv-maxrate 25000 --b-pyramid none --ref 5 --stdin y4m --output NUL -"
y4m [info]: 1280x720p 1:1 @ 25/1 fps (cfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x264 [info]: OpenCL acceleration enabled with Advanced Micro Devices, Inc. Capeverde (SI)
x264 [info]: profile Main, level 4.1, 4:2:0, 8-bit
A s s e r t i o n   f a i l e d !

 P r o g r a m :   \ \ H O S T \ R i p b o t 2 6 4 t e m p \ t o o l s \ x 2 6 4 \ x 2 6 4 _ x 6 4 . e x e
 F i l e :   e n c o d e r / s l i c e t y p e . c ,   L i n e   1 9 8 9

 E x p r e s s i o n :   c o s t   > =   0
Can someone give me a hint how i can solve this problem?
Thanks
userx is offline   Reply With Quote
Old 5th August 2019, 19:38   #17077  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
Are you sure you are using latest version?

I'm asking because this looks odd
Quote:
--opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead.clbin"
It should be with digit related with server instance
Quote:
--opencl-clbin "C:\Users\UserX\AppData\Local\Temp\x264_lookahead_1.clbin"

Last edited by Atak_Snajpera; 5th August 2019 at 19:42.
Atak_Snajpera is offline   Reply With Quote
Old 6th August 2019, 05:50   #17078  |  Link
userx
Registered User
 
userx's Avatar
 
Join Date: Mar 2016
Location: Austria
Posts: 28
Quote:
Originally Posted by Atak_Snajpera View Post
Are you sure you are using latest version?

I'm asking because this looks odd

It should be with digit related with server instance
yes, it's the latest version 1.25
in ...\AppData\Local\Temp\ i found
x264_lookahead.clbin
x264_lookahead_1.clbin
x264_lookahead_2.clbin
x264_lookahead_3.clbin
All with file changed date from yesterday

Does it matter if i start Ripbot264.exe on remote machine which start EncodingServer instances automatically compared to starting EncodingServer instances alone?

Last edited by userx; 6th August 2019 at 05:55.
userx is offline   Reply With Quote
Old 6th August 2019, 14:02   #17079  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,594
Can you upload all .cmd files from Chunks folder?
Are you using multiple servers on the same machine? If yes then does this problem also occur if you use only one?
Atak_Snajpera is offline   Reply With Quote
Old 6th August 2019, 17:50   #17080  |  Link
brumsky
Registered User
 
Join Date: Jun 2016
Posts: 116
Hey Atak_Snajpera,

Would it be possible to configure a way to use analysis save and load while doing distributed encoding? Thinking through it you'd have to code a way to save a separate analysis file for each chuck and configure a way to reuse them.

thanks for all your hard work.
brumsky is offline   Reply With Quote
Reply

Tags
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.