RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY

[mparade]

All jobs will use the same server configuration.


So now you know what to do ;) BTW. Overclocking AMD cpus do not make sense these days.

There should be something else here...After switching back to auto mode in amd ryzen master, Windows still keeps resetting very frequently right in the middle of encoding...temperature is below 68 degrees so that shouldn't be the problem
maybe memory issue or something else :)

[mparade]

Your admin should know what to do.
https://www.youtube.com/watch?v=Mk_XYuzQ8jA

Ports to open
https://social.technet.microsoft.com/Forums/Azure/en-US/0fff3b48-7615-45ea-817e-5afaad972c43/vpn-ports-to-open?forum=winservergen

thank you very much

[Atak_Snajpera]

thank you very much

This one is even better
https://www.youtube.com/watch?v=eVxHhSLUrxo

[mparade]

All jobs will use the same server configuration.


So now you know what to do ;) BTW. Overclocking AMD cpus do not make sense these days.

Without overclocking it does the same so maybe voltage drop occurs...
so maybe I need to replace the likely faulty 1200W Cooler Master PSU first. :)

[skylinekiller]

so is it no longer possible to change the subtitle fonts.

I am trying to edit the font when I hardcode subtitle into my encode, but it doesn't seem to be working. I edited the notepad file in Tools\AviSynth plugins\VSFilter\Default.style with the parameters below.

Style: Default,Univers CE 45 Light,41,16777215,0,16777215,0,-1,0,2,1,2,20,20,20,0,0

what it looks like with the above settings in RipBot https://ibb.co/S7xFvpn


This is what it should look like. The resolution is not as good but trusts me, the font is not accepting the settings
https://ibb.co/92VB0Xt


This is the setting from srt2ssa

Synch Point: Side 1 0m00s
Collisions: Normal
Timer: 100.0000
Style: Default,Univers CE 45 Light,41,16777215,0,16777215,0,-1,0,2,1,2,20,20,20,0,0


Why am I getting a shadow

[mparade]

Possibly yes. Your x264 build does not support hardware acceleration (if there is a speed-up at all), but that doesn't keep it from finishing its job correctly with usual CPU routines only.

There may be x264 builds out there which can use some OpenCL functions of a modern GPU. So you might replace xour x264 executable; but I won't guarantee that they work correctly with RipBot264, they might interfere with other filters using the GPU too, even. Test and get experienced.

Can anyone recommend a reliable site for x264 builds using OpenCL functions? I have a 2060 RTX so just want to give it a try if possible.

Thank you in advance for the help.

[sneaker_ger]

Try:
https://artifacts.videolan.org/x264/ (no mp4 output - is that a problem for RipBot?)

[mparade]

thx! sorry for the dumb question but if I have Win 64 bit and AMD CPU which version to use AMD64 or Win64 or all the same?

[sneaker_ger]

"release-debian-amd64" is for for the Linux distribution "Debian", not Windows. Try "release-win64" (some software may need "release-win32" for compatibility but win64 is faster and can use more RAM so try that first).

[mparade]

I test is very soon, thank you!

[mparade]

"release-debian-amd64" is for for the Linux distribution "Debian", not Windows. Try "release-win64" (some software may need "release-win32" for compatibility but win64 is faster and can use more RAM so try that first).

Using this version I got the same disabling message from x264: x264 [warning]: OpenCL: not compiled with OpenCL support, disabling

[sneaker_ger]

I tested opencl works with https://artifacts.videolan.org/x264/release-win64/x264-r2984-3759fcb.exe
So you are still using the "wrong" binary somehow.

[mparade]

I have double checked. Ripbot is using the same from it's temp directory. Maybe the commandline made by Ripbot is not ok for the encoder. I don't really know what is happening.

Maybe it has something to do with video decoding? I am using LSMASH.

Thanks for any kind of help.

[LigH]

thx! sorry for the dumb question but if I have Win 64 bit and AMD CPU which version to use AMD64 or Win64 or all the same?

Based on the IA-32-x86 architecture (which was introduced with the intel 80386 CPU), AMD and intel both developed an extension to allow 64 bit addressing. intel first started with a new architecture known as IA-64 (Itanium) which is not compatible with previous software. AMD instead developed a compatible extension x86-64, and intel realized how useful it is to continue using existing 32-bit software while developing 64-bit software with mostly the same instruction set for the future, and developed AMD's x86-64 architecture further with own extensions...

So if you don't have specifically an Itanium Server software for Windows running on an Itanium CPU, which I doubt a lot, then you will have a 64-bit Windows running on a CPU compatible to the AMD64 or x86-64 instruction set architecture. But there are also other operating systems running on such processors, it doesn't have to be Windows.

Talking about a 64-bit Windows, you can see Win64 and AMD64 and x86-64 (sometimes even x64) as more or less equal. As long as it does not explicitly say IA-64 / Itanium.

[Atak_Snajpera]

I have double checked. Ripbot is using the same from it's temp directory. Maybe the commandline made by Ripbot is not ok for the encoder. I don't really know what is happening.

Maybe it has something to do with video decoding? I am using LSMASH.

Thanks for any kind of help.

x264 included in ripbot264 HAS support for OpenCL. Tested on crappy Geforce GT 710 and it works fine. Problem is with your drivers.

[mparade]

Thank you. I am using latest NVIDIA Game Ready Driver.

[sneaker_ger]

@mparade
Are you using 10 bit encoding? It seems x264 doesn't support opencl+10 bit and then throws this confusing warning about "not compiled with OpenCL support".

[mparade]

1 minute ago I realized it too. :). Thank you very much! In 8-bit it was working like a charm. I have to think over then if I really want 10bit encoding. All of my players support 10bit files, a bit slower encoding times but with a 2990WX...:)

[mparade]

Is it possible to have jobs deleted automatically after they are completed successfully? (Not in batch mode)

Thank you!

[Atak_Snajpera]

Sure.

[mparade]

May I ask where? In batch window I have found this option but nowhere else, please help.

Thank you in advance.

Edit: just found it...sorry for the question

[mparade]

Sometimes I have problems with dtsma sources.
Sometimes they recognized with faulty channel numbers, sometimes they are demuxed and after being chosen from the list relevant parameters and even profiles are not loaded at all. So I have to ignore them as a source stream. Is there a solution for this yet?

[SKPN]

Well, as I said, I'd like to have my 4K's at a "standard" of 60Mps, and again, as I've said, some have lower than that, some are higher, so for that I need to run them thru RB, even if it's video only.

This part makes no sense. You shouldn't be aiming for a specific bit rate unless you have a limited amount of space, and even then, should only mean lowering the bit rate. Encoding at a higher bit rate than the source video won't increase the quality, so there is no point in doing so; all you're doing is wasting space.

One thing I would like to get around to doing is upscaling some 1080p to 4K, but do any filtering, etc, still @ 1080p, then once satisfied, do a 4K HEVC encode, and see how it turns out.

This is not a good idea. The more you encode a video, the worse the quality will be. So if you take the 1080p source, apply filters and encode it, then take the encoded video and encode it again at 4K, it will look worse than doing both the upscale and filters in a single encode. Every time you re-encode a video, you're losing data.

[guest]

I noticed another small auto update today (just core.exe). A change to the command script, cmd features/control.

A lot of time seems to be getting spent on this script for when a job is completed...I for one don't really get the idea behind this, I think someone requested a feature for deleting completed jobs.....

If that's basically what it's for, I would prefer to do it manually !!

And what of my simple feature request, on RB gui's main window, the current core version/build is displayed (for quick reference)

[guest]

I will dig out a "crappy ssd", and try a "big ass 80Gb" encode, with the latest build of RB, and see just how long it does take....

If it still takes long time then I will rethink your request. Just give me some hard numbers.

Just a little update on my "big ass" encode....

I was only about an hour from it starting the muxing process, and the SSD I was using for the Temp files, ran out of space.

It was "only" 120Gb, and with the ONE movie to process, filled it up :(

So will have to re-think the setup, and start all over again :(

Regardless of that problem, I noticed that earlier on during the chunks being processed, that randomly a chunk would just "reset", and start again from 0%, so all it had processed up to that point, was lost. :(

So I turned off the re connect option, and it seemed to be OK, so something in the workings of that, seems to create a problem, unless there's a brief hiccup in the network connection.

[guest]

Would it be too much to ask if you could implement an option where you could send a shut down/kill command thru to an individual server's IP address, when ever you wanted or needed to ??

Preferably within the Encoding Client window/screen.

Very handy when the servers are elsewhere !!

Separate to the shut down options at the end of an encode.

[byteshare]

Well, as I said, I'd like to have my 4K's at a "standard" of 60Mps, and again, as I've said, some have lower than that, some are higher, so for that I need to run them thru RB, even if it's video only.
For what I want & need to do to audio, can be dealt with other app's, as too, the subtitles.
I have to agree, I don't like grain either, but fortunately most 4K's aren't very "noisy", so not a lot of filtering required.
I know I need to try a few different options, but it's the time factor that's my main concern...imagine doing a high bitrate 4K HEVC, on a Slow preset, using MDGrain2....day's :( even with my "farm".
But the other time waster is between when the encode is completed, and the start of the next job in queue...meaning the muxing, etc...it leaves a lot of resources doing nothing. :(, but having said that, I am yet to test this again, after putting some SSD's in the mix.
One thing I would like to get around to doing is upscaling some 1080p to 4K, but do any filtering, etc, still @ 1080p, then once satisfied, do a 4K HEVC encode, and see how it turns out.
But just imagine the time & resources involved when 8K is a lot more common place.
You'd want to look at some of the advanced x265/HEVC commands to preserve detail but you should still be able to reduce size while it being near same quality.
Personally I go for looks 95% precieved quality, so I don't have any good settings for you to start with.
That said, if you are using filters like MDGrain2, IMO you shouldn't worry about the bitrate because you're going to be chopping out a lot with that so you should just use CRF12 or so from what it I'm guessing you like. If you are using constant bitrate with MDegrain2 and it is set higher/same as source you're wasting bits.
If you're going to use constant bitrate without filters I'd set it a little lower than source at least and do 2pass and you'll likely get something 99.9999% the same if you can even see a difference (depending on your settings). Some HEVC settings do bluring, slight denoise/dehalo type stuff so if it isn't looking close to the source your settings are the issue.
From the sounds of it you need a bigger SSD, guessing at least 500GB or more.
As for upscaling the way you want to do it isn't a problem since you'd do the filters either before or after the upscale and upscale all in one job, then encode.

I noticed another small auto update today (just core.exe). A change to the command script, cmd features/control.
A lot of time seems to be getting spent on this script for when a job is completed...I for one don't really get the idea behind this, I think someone requested a feature for deleting completed jobs.....
If that's basically what it's for, I would prefer to do it manually !!
And what of my simple feature request, on RB gui's main window, the current core version/build is displayed (for quick reference)
I just got a chance to use the new update and it is great for me. Really reduced my manual tasks.
You can do a lot with it besides deleting source files.
Also, made sure when a lot of jobs fail my drives won't fill up.

Is it possible to have jobs deleted automatically after they are completed successfully? (Not in batch mode)
Settings > [uncheck] Keep jobs after successful conversion

[Dhry]

Okay, so today Ripbot updated core (2019.10.15), tried an encode of an MP4 file via adding to batch, but all it does is show "Waiting for file..." in the main window and nothing happens after that. Usually this is the point at which it demuxes etc. I tried it on multiple mp4s, always the same. Looking at the temp folder, the job1 folder is created but no work files are added to it.

[Atak_Snajpera]

Okay, so today Ripbot updated core (2019.10.15), tried an encode of an MP4 file via adding to batch, but all it does is show "Waiting for file..." in the main window and nothing happens after that. Usually this is the point at which it demuxes etc. I tried it on multiple mp4s, always the same. Looking at the temp folder, the job1 folder is created but no work files are added to it.
Waiting for file... means in practice waiting for file to be accessible. Are you sure nothing is blocking your files? I've just tested on my machine and everything works fine. Besides I didn't touch anything in code regarding this issue.

[Dhry]

Waiting for file... means in practice waiting for file to be accessible. Are you sure nothing is blocking your files? I've just tested on my machine and everything works fine. Besides I didn't touch anything in code regarding this issue.

Definitely not. I thought the same thing as well, and used Unlocker on each file to make certain that something had not blocked access to them. They were not opened for reading or writing by anything. I'll reboot my machine in a little while and will try again. Cheers!

[byteshare]

Definitely not. I thought the same thing as well, and used Unlocker on each file to make certain that something had not blocked access to them. They were not opened for reading or writing by anything. I'll reboot my machine in a little while and will try again. Cheers!
Did you make the job after the update, or before?
If before try deleting the job and remaking it.
After the reboot, if you're still having issues try posting the media info.

[Dhry]

Did you make the job after the update, or before?
If before try deleting the job and remaking it.
After the reboot, if you're still having issues try posting the media info.

Job was made after the update, with the shared job folder completely and absolutely empty. Deleted job and remade it, same issue. Deleted job, deleted job1 folder in temp folder, created again, same. Ripbot has worked in this configuration through multiple updates, but this last update is broken somehow. I started a batch encode and added a single file, this time from a completely different folder, absolutely nothing locking the file. Even if I click Abort Job Creation, it tells me "please wait.. aborting" and that never ends. I'm using a test mp4 file from one of my video cameras. Media info is as follows, hope this is the format you're asking for:

General
Complete name : K:\test.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 28.7 MiB
Duration : 39 s 0 ms
Overall bit rate mode : Variable
Overall bit rate : 6 166 kb/s
Encoded date : UTC 2018-08-04 00:02:07
Tagged date : UTC 2018-08-04 00:02:07

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 1 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 1 frame
Format settings, GOP : M=1, N=30
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 39 s 0 ms
Bit rate mode : Variable
Bit rate : 6 162 kb/s
Maximum bit rate : 30.7 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 30.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.099
Stream size : 28.6 MiB (100%)
Encoded date : UTC 2018-08-04 00:02:07
Tagged date : UTC 2018-08-04 00:02:07
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC

Screenshot of my batch settings attached.

EDIT: I added this file as normal (ie not through the Batch mode) and it worked. Normal add probes, demuxes, indexes etc as normal. Distributed encoding works great. Problem seems to be within the batch functionality.

[guest]

If it still takes long time then I will rethink your request. Just give me some hard numbers.

OK, here we go, this is what I observed, when encoding my "big ass" 4K, 60Mb/s bitrate encode.

Now I know a lot here cannot understand why I'm doing this the way I'm doing, (but that's my choice) so with that aside, what I have documented could impact nearly anyone that is doing 4K HEVC encodes, using Distributed Encoding.

So first things, I had an approx 65Gb, @ approx 65Mb/s bitrate movie (John Wick 3).

I queued it up, the initial demuxing & indexing of the file, took 27 minutes, using Lsmash.

Then the next day, when I started the job, it took a further 15 minutes, before it started encoding, and initiating the DE Servers, and I am ONLY doing the video.

So over the next 3 days (3 different encoding session's), probably turned out to be around 12 - 14 hour's using varying different server's in the "encoding farm".

Just a foot note, I have a lot of Solar Panels on the roof, and I won't do any long term processing jobs unless I can get it for "free", so that can change from day to day.

So, finally, late today, the chunks were all done.

So I got the phone out, and started the stop watch, and the combining of the chunks took 10 minutes 30, then the muxing process took a further 15 minutes 30, using SSD's for the Temp folder & where the encode is compiled.

I would suggest that this time would be shorter if the bitrate wasn't as high as I had it (1 Pass @ 60,000kb/s), ended up being 59.6Mb/s (Mediainfo).

I am yet to mux back in the audio (which I have changed from TrueHD to DTS-MA), and a subtitle track (.srt)

So, getting back to a question raised a week or so ago, to have RB to start the next job in the queue, whilst it is combining the chunks, and the final muxing. (Is muxing a single threaded process ?)

So observations of "my" process, there is approx 45 minutes where NONE of the DE Server's are being used, when starting a new 4K job, from scratch....then of course the DE Server's are going flat out 'til all the chunks have been completed.

Then, hopefully the Client PC is the one left doing the final combining & muxing, which leaves ALL the DE Servers doing nothing for a further 25 or so minutes. So they could be turned off until the next job has actually start encoding, for just over an hour. (Between jobs)

So as great as DE is, there can be a LOT of time that the servers aren't doing anything..so if there was a way to have RB to at least start the next job, once the chunks of the previous job were completed, that could save a reasonable amount of time, and power.

OR,

if there was someway for RB to start the DE servers once the job is about to start encoding, (when the chunks are being created) and maybe shutdown the DE servers once the chunks are done, instead of once the full encode was completed.

OR,

some option to manually turn the remote DE servers on & off from the Client pc.

[Atak_Snajpera]

So I got the phone out, and started the stop watch, and the combining of the chunks took 10 minutes 30, then the muxing process took a further 15 minutes 30, using SSD's for the Temp folder & where the encode is compiled.
~100 MiB/s copy speed on SSD<->SSD? Are you using TLC SSD? I'm asking because direct write to TLC NAND is around that level. QLC is even worse (~75MiB/s)

[guest]

~100 MiB/s copy speed on SSD<->SSD? Are you using TLC SSD? I'm asking because direct write to TLC NAND is around that level. QLC is even worse (~75MiB/s)

WD Greens in Raid 0 for Temp, some Chinese SSD for the Encodes.

That's all I had, spare.

[Ryushin]

Why not just use MakeMKV. Make a MKV with just the video and the audio you want. Then extract the subtitles out, OCR and convert them to SRT/ASS/etc, mux them back in, and your done. Your video quality will be exactly the same as the source. Most smaller than 60Mb/s and some a bit larger. Your finished file will done quicker than ever having to encode. From what I understand, unless you need to degrain, there is no need to re-encode the source.

When I encode, I use CRF18 for x265 and I cannot tell the difference between the original source and my output. Some 4K sources that have a huge amount of grain such as Blade Runner, come out looking far better than the original when using MDegrain2/3.

[Atak_Snajpera]

WD Greens in Raid 0 for Temp, some Chinese SSD for the Encodes.

That's all I had, spare.

Could you test your SSD/HDD in my benchmark tool?
http://www.mediafire.com/file/i630b6n1fal38fj/IOSpeedTester.7z/file

Results on my 5 years old 120GB SSD (MLC)
https://i.imgsafe.org/65/652bfb9795.png

It is a command line tool. Use these switches

IOSpeedTester.exe --test-location "X:\" --transfer-size 120GIB --log-file "C:\users\%username%\Desktop\MyLog.csv"


Where X is a drive letter of your drive.

Upload MyLog.csv somewhere.

full help
IOSpeedTester v1.0 by Atak_Snajpera
Syntax: IOSpeedTester [options]

Options:
--help Show this help text and exit
--test-location <string> Folder where test files will be stored
--threads <1..64> Number of working threads. [1]
--buffer-size <1KiB..64MiB> Write/Read buffer size. [16MiB]
--transfer-size <integer> Amount of bytes to transfer. [1GiB]
--buffered-io Enables IO buffering in RAM. [Unbuffered]
--log-file <string> Log information to specific file. [None]
--write-only Performs write test only. [Write+Read]

Example usage:

Sequential Test with logging to a file
IOSpeedTester --test-location "%temp%" --log-file "MyLog.csv"

MIN IOPS Test
IOSpeedTester --test-location "%temp%" --buffer-size 4KiB

MAX IOPS Test
IOSpeedTester --test-location "%temp%" --buffer-size 4KiB --threads 64

[byteshare]

EDIT: I added this file as normal (ie not through the Batch mode) and it worked. Normal add probes, demuxes, indexes etc as normal. Distributed encoding works great. Problem seems to be within the batch functionality.
Batch function is working for me on 2 different systems...I'm not sure what is happening for you. I'll try to think on it.

[byteshare]

I was trying to encode some HDR and I normally don't define the colorprim, transfer, or colormatrix. I also don't use the tone mapping.
I'm getting the error: x265 [error]: invalid argument: transfer = bt.2020
With tone mapping I don't, as I'd expect.
I tried defining the colorprim, transfer, and colormatrix:
--colorprim bt2020 --transfer bt2020-12 --colormatrix bt2020c
Because it looked like RB was doing it wrong (media info at the bottom), with instead:
--colorprim bt2020 --transfer bt.2020 constant --colormatrix bt2020c
But I'm seeing both RB and my settings in the the Encoding Client:
Encoding started...
""\\SERVER\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\SERVER\RipBot264temp\job36\Chunks\1.avs" -strict -1 -f yuv4mpegpipe -
| "\\SERVER\Ripbot264temp\tools\x265\x265_x64.exe" --seek 0 --colorprim bt2020 --transfer bt.2020 constant --colormatrix bt2020c --crf 22 --fps
24000/1001 --min-keyint 24 --keyint 240 --frames 1405 --sar 1:1 --profile main10 --output-depth 10 --colorprim bt2020 --transfer bt2020-12
--colormatrix bt2020c --aq-mode 3 --ctu 64 --y4m --pools "+" --output "\\SERVER\RipBot264temp\job36\Chunks\1.265" -"
x265 [error]: invalid argument: transfer = bt.2020

Is this a RB thing or what am I doing wrong?

Media Info:
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L6@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 48 min 21 s
Bit rate : 64.5 Mb/s
Width : 4 096 pixels
Height : 2 304 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.285
Stream size : 21.8 GiB (100%)
Writing library : x265 2.4+27-e9e574bbed93:[Windows][GCC 6.3.0][64 bit] 10bit
Encoding settings : cpuid=1173503 / frame-threads=2 / numa-pools=4 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 /
input-csp=1 / input-res=4096x2304 / interlace=0 / total-frames=69568 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance
/ no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / bframes=8 /
b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect
/ no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip
/ nr-intra=0 / nr-inter=0 / no-constrained-intra / no-strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 /
merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=-3:-3 / no-sao / no-sao-non-deblock / rd=4 / no-early-skip
/ rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / rdpenalty=0 / psy-rd=0.30 / psy-rdoq=2.50 / no-rd-refine / analysis-mode=0
/ no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=16.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30
/ aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / sar=0 / overscan=0
/ videoformat=5 / range=0 / colorprim=9 / transfer=15 / colormatrix=10 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 /
max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps
/ scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / no-hdr-opt / no-dhdr10-opt / refine-level=5 / no-limit-sao / ctu-info=0
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : BT.2020 (12-bit)
Matrix coefficients : BT.2020 constant

(added linebreaks for scrolling reduction)

[Atak_Snajpera]

Send me 100MiB sample with that BT.2020 (12-bit) transfer characteristics.

[slalom]

So as great as DE is, there can be a LOT of time that the servers aren't doing anything..so if there was a way to have RB to at least start the next job, once the chunks of the previous job were completed, that could save a reasonable amount of time, and power.
But Temp disk is being used for the previous job, you will slow it down if the program starts a new job simultaneously

[byteshare]

But Temp disk is being used for the previous job, you will slow it down if the program starts a new job simultaneously
Good point. If you're muxing on the temp drive while trying to transfer the video to the job folder at the same time that would only compound things.
You'd have to pre-load the video for the next job once the current job starts to get around that but still not starting while the last job is muxing...tricky timing.

[guest]

Good point. If you're muxing on the temp drive while trying to transfer the video to the job folder at the same time that would only compound things.
You'd have to pre-load the video for the next job once the current job starts to get around that but still not starting while the last job is muxing...tricky timing.

You're probably correct, but at least while the muxing is being done, the next job is at least starting it's process, which could save some time & resource's.

Even if the muxing & copying was a little slower, you're not having to wait for one job to be completed.

But here's an idea, what if there was a second temp drive, and you allocated which temp drive to be used, when creating new jobs. (wouldn't work too well with batching)

[byteshare]

You're probably correct, but at least while the muxing is being done, the next job is at least starting it's process, which could save some time & resource's.
Even if the muxing & copying was a little slower, you're not having to wait for one job to be completed.
That is kind of the point, it would slow it down enough that your next job would actually be encoding later than the way it works now. Yes, technically the job would "start" sooner but the time for the video to copy to the temp drive would take longer with the last job muxing at the same time.
The only way it would be "faster" is if once jobs are started the next job loads the video while the other job is still encoding, then muxing would happen and the next job would start, then while that job is still going but after the last muxing is done the next-next job would start copying the video...this only really works if your jobs are both small enough/slow enough encodes that the timing can work...like I was saying, it would be tricky and wouldn't be helpful in many cases.
...Ideally if some symbolic link to the original file could be used for the sharing (in DE) so that you wouldn't have to even copy a file, then it would be much simpler/faster to start the next job while the last job is muxing.
But here's an idea, what if there was a second temp drive, and you allocated which temp drive to be used, when creating new jobs. (wouldn't work too well with batching)
Wouldn't work for a lot of people I'd guess. Wouldn't work for me. I'd have to get a 3rd drive.

[guest]

That is kind of the point, it would slow it down enough that your next job would actually be encoding later than the way it works now. Yes, technically the job would "start" sooner but the time for the video to copy to the temp drive would take longer with the last job muxing at the same time.
The only way it would be "faster" is if once jobs are started the next job loads the video while the other job is still encoding, then muxing would happen and the next job would start, then while that job is still going but after the last muxing is done the next-next job would start copying the video...this only really works if your jobs are both small enough/slow enough encodes that the timing can work...like I was saying, it would be tricky and wouldn't be helpful in many cases.
...Ideally if some symbolic link to the original file could be used for the sharing (in DE) so that you wouldn't have to even copy a file, then it would be much simpler/faster to start the next job while the last job is muxing.

Good to see you've given this some thought, as well.

So I have to ask, do you do many, if any, 4K movie encodes, and how many pc's can you use for DE ??

I'm just trying to figure out some way to utilise the DE servers more energy efficiently. It just frustrates me to see several servers whirring away, using power, but not doing any encoding :(

Wouldn't work for a lot of people I'd guess. Wouldn't work for me. I'd have to get a 3rd drive.

Well, if I step up to using SDD's or NVMe's for the temp & encodes drives, I'm going to have to buy a LOT :(

But if a 2nd temp drive could be implemented, I think it would be worth having to buy another drive :)

[guest]

Could you test your SSD/HDD in my benchmark tool?
http://www.mediafire.com/file/i630b6n1fal38fj/IOSpeedTester.7z/file

Upload MyLog.csv somewhere.

OK, have done your test's as requested, I hope I have provided the info you wanted to know.

I checked ALL the drives in that PC, just for comparison, some didn't have enough free space to do the 120Gb test, but the speeds are still relevant, I'd say.

Now remembering that these SSD's aren't anything special, they were just a few I had lying around.

http://www.mediafire.com/file/dqlf6cqxspncodu/Desktop.rar/file

[byteshare]

Good to see you've given this some thought, as well.

So I have to ask, do you do many, if any, 4K movie encodes, and how many pc's can you use for DE ??

I'm just trying to figure out some way to utilise the DE servers more energy efficiently. It just frustrates me to see several servers whirring away, using power, but not doing any encoding :(
I do some 4K. I've been doing more and more with time.
I have had up to 6 computers in DE mode, but right now I use 2 since upgrading some systems (made the others ones pointless), and 1-2 other computers with a local only DE mode. I see no reason you can't easily have
8-16 computers being used on one jobs (assuming longer/high CPU jobs).
The main thing is having a fast temp drive for large files.
Well, if I step up to using SDD's or NVMe's for the temp & encodes drives, I'm going to have to buy a LOT :(

But if a 2nd temp drive could be implemented, I think it would be worth having to buy another drive :)

The main thing would be using the SSD/M.2 for the temp drive (locally mux) then if you still want it somewhere else, have it auto-move to another drive with the new after job scripting to allow the next job will start while it is moving.

[Atak_Snajpera]

OK, have done your test's as requested, I hope I have provided the info you wanted to know.

I checked ALL the drives in that PC, just for comparison, some didn't have enough free space to do the 120Gb test, but the speeds are still relevant, I'd say.

Now remembering that these SSD's aren't anything special, they were just a few I had lying around.

http://www.mediafire.com/file/dqlf6cqxspncodu/Desktop.rar/file

Your SSDs are even worse than your HDDs! :eek:
No wonder that muxing takes ages! You really need some good MLC SSD instead of those TLC garbage.

https://i.ibb.co/z4vWySb/ssd-samsung-evo-840.png

https://i.ibb.co/5WSSQW0/SSD-Veseky.png

https://i.ibb.co/D4wCVfc/hdd-Seagate.png

https://i.ibb.co/vVV6Ty7/hdd-Wd-green.png

[guest]

Your SSDs are even worse than your HDDs! :eek:
No wonder that muxing takes ages! You really need some good MLC SSD instead of those TLC garbage

Not that surprised by that...I did say they were all I had...

So, what would you suggest...SSD's or PCI-e mounted NVMe's ??

[byteshare]

Your SSDs are even worse than your HDDs! :eek:
No wonder that muxing takes ages! You really need some good MLC SSD instead of those TLC garbage.
Wow, I've never seen that. Maybe they're also near the end of their life?
TY for the graphs.

So, what would you suggest...SSD's or PCI-e mounted NVMe's ??
Those or M.2. Pretty much any drive that does 500MB/s or more read/write, which is pretty common these days.
I have an M.2 that does around 2GB/s for read/write.