Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th December 2019, 13:47   #18101  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Is it a problem at my end, or is it something that you need to dig a little deeper into ??
This has nothing to do with my code. My code just executes .cmd file and then waits for spawned ffmpeg.exe and x264.exe/x265.exe. When they terminate my code next checks number of encoded frames and reports to encodingclient with proper command. Nothing extraordinary. I would try not to push CPU usage to absolute 100%.

Last edited by Atak_Snajpera; 26th December 2019 at 13:49.
Atak_Snajpera is offline   Reply With Quote
Old 26th December 2019, 14:04   #18102  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by Atak_Snajpera View Post
This has nothing to do with my code. My code just executes .cmd file and then waits for spawned ffmpeg.exe and x264.exe/x265.exe. When they terminate my code next checks number of encoded frames and reports to encodingclient with proper command. Nothing extraordinary. I would try not to push CPU usage to absolute 100%.
OK, but what if just 1 DE server uses 100%....how could it be cut back a little ??

Would a newer ffmpeg make any difference ??
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 26th December 2019, 14:09   #18103  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Originally Posted by Pauly Dunne View Post
OK, but what if just 1 DE server uses 100%....how could it be cut back a little ??

Would a newer ffmpeg make any difference ??
I mean ,do not run extra server if you already have 90%+ cpu usage. Another thing. Run EncodingServer in LOW priority instead of NORMAL.

Last edited by Atak_Snajpera; 26th December 2019 at 14:15.
Atak_Snajpera is offline   Reply With Quote
Old 26th December 2019, 14:17   #18104  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by Atak_Snajpera View Post
I mean ,do not run extra server if you already have 90%+ cpu usage.
Well, I noticed today (yesterday now), that with the custom MDegrain3 filter I'm using, REALLY max's everything out, and just 1 DE server per PC or NUMA node, pretty much ran @ 100%

Just saw your edit, so I will run at LOW, and see what that does
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)

Last edited by Pauly Dunne; 26th December 2019 at 14:25.
Pauly Dunne is offline   Reply With Quote
Old 26th December 2019, 14:24   #18105  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
BTW. If you want to check if UMA is slower/faster than NUMA then you should compare speed only using that one PC. Do not look at some random chunks because some parts of the movie are just easier to compress (faster encoding speed).
Atak_Snajpera is offline   Reply With Quote
Old 26th December 2019, 14:31   #18106  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by Atak_Snajpera View Post
BTW. If you want to check if UMA is slower/faster than NUMA then you should compare speed only using that one PC. Do not look at some random chunks because some parts of the movie are just easier to compress (faster encoding speed).
I think I'll stay with using NUMA.

That random test (in that screenshot), was a reasonable difference, even taking in to account what you said about different parts of a movie being easier to compress.
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 26th December 2019, 14:37   #18107  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
NUMA may be faster due to direct access to memory. In UMA threads have to communicate between sockets which is obviously much slower (higher delay and so on).
https://techdifferences.com/differen...-and-numa.html
Atak_Snajpera is offline   Reply With Quote
Old 27th December 2019, 00:47   #18108  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by Atak_Snajpera View Post
Do not abort. Instead wait for encoding server to finish encoding. There is high chance that at the end progress on encondingclient side will immediately change to encoded ...

Like I predicted...

Most likely ffmpeg.exe (decoder process) silently terminated prematurely because it couldn't get data from host machine.
I forgot to mention that I DID see a ffmpeg error pop up message on one of my servers that I'm running W8.1 on.

Something like "ffmpeg stopped responding" (or similar)
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 27th December 2019, 06:58   #18109  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by Atak_Snajpera View Post
I mean ,do not run extra server if you already have 90%+ cpu usage. Another thing. Run EncodingServer in LOW priority instead of NORMAL.
Well, yet another day of "testing", but there has been no related problems today.

I had one hiccup which I think was network related (posted error msg)

https://www.mediafire.com/file/qpbfl...ppage.txt/file

And to my surprise there was an Autoupdate waiting, so that was great.

I didn't update my Client PC however, just in case it jeopardised the unfinished job from yesterday, didn't want that to go to waste.

Updated ALL the servers, changed the CPU Priority down to LOW.

So once I started yesterday's job, I slowly introduced the servers, noting their encoding speed's.

I also noticed that trying to start 2 servers on NUMA 0 & 1, didn't seem to work all that well, so I just used NUMA 0,
but then I noticed that basically only 1 CPU was working, and even the cooling radiators proved that one was working, the other was not.

So due to that, I progressively changed my older SMC boards back to UMA, and changed the priority to NORMAL and VARIABLE, and ran 1 DE server on each pc.

Oh, and that had the CPU's just "bouncing off" 100%....

Seemed to work quite well, both rad's were warm, encoding speed was up, so maybe thats the way to go, when giving them a heavy filtered encode to process.

But having said all that, would there be a way in Encoding Client, that when one chunk is complete (that was using say, NUMA 0), that the next chunk on that IP would switch over and use NUMA 1 (just to share the load over both CPU's during the encode, not just one doing all the work) ??

I might also try to use 2 different RipBot folders, and have one that was configured to these heavy filtering jobs, and used certain DE servers.

And have another, setup to use more and/or different DE servers & settings, for "lighter" loads.

So all in all, a pretty successful day (hopefully tomorrow is as good)
_________________________________________________________________________________________________

A couple question's about the update....I noticed "HDRtools"....how is that used / enabled ??

Also, under "Default Video Decoder", it's changed from Lsmash, FFMS2 & GPU, to just CPU & GPU ??

So is Lsmash the one and only, now ??
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 27th December 2019, 17:10   #18110  |  Link
stryker412
Registered User
 
Join Date: Feb 2002
Posts: 151
Just finished my new build. Are there any settings I should enable/disable with this new system?

Ryzen 3700X
16GB RAM
1TB NVMe (Adata SX8200 Pro)
Nvidia RTX2070 Super

I'm testing with an UHD movie now. I set it to 10bit, CQ 18. Looks like it's going to take around a total of 6hrs. Does that sound right?
stryker412 is offline   Reply With Quote
Old 27th December 2019, 17:21   #18111  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Originally Posted by stryker412 View Post
Just finished my new build. Are there any settings I should enable/disable with this new system?

Ryzen 3700X
16GB RAM
1TB NVMe (Adata SX8200 Pro)
Nvidia RTX2070 Super

I'm testing with an UHD movie now. I set it to 10bit, CQ 18. Looks like it's going to take around a total of 6hrs. Does that sound right?
GPU decoding may give you few percent speed boost...
Atak_Snajpera is offline   Reply With Quote
Old 28th December 2019, 01:16   #18112  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by stryker412 View Post
Just finished my new build. Are there any settings I should enable/disable with this new system?

Ryzen 3700X
16GB RAM
1TB NVMe (Adata SX8200 Pro)
Nvidia RTX2070 Super

I'm testing with an UHD movie now. I set it to 10bit, CQ 18. Looks like it's going to take around a total of 6hrs. Does that sound right?
Hey stryker412,

That's great you have a nice new system up and running (slightly envious).

So, tell us a little more about your "test" encode...

Whats the movie ??

What other settings are you using, filters, are you able to use DE ??

It all makes a difference.
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 28th December 2019, 04:02   #18113  |  Link
stryker412
Registered User
 
Join Date: Feb 2002
Posts: 151
I used Wreck it Ralph since it was a shorter film. I didn't change any settings other than bump CQ from 20 to 18. The final file was only 4.8GB which seems a little small for an HD10 movie. I also only used the DD track as it seems TrueHD has issues on my system. Plus, I only have a 5.1 setup anyway.

Is GPU decoding only in the ini file settings? I didn't see it in the GUI settings.
stryker412 is offline   Reply With Quote
Old 28th December 2019, 04:31   #18114  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by stryker412 View Post
I used Wreck it Ralph since it was a shorter film. I didn't change any settings other than bump CQ from 20 to 18. The final file was only 4.8GB which seems a little small for an HD10 movie. I also only used the DD track as it seems TrueHD has issues on my system. Plus, I only have a 5.1 setup anyway.

Is GPU decoding only in the ini file settings? I didn't see it in the GUI settings.
So is that 1080p or 2160p (4K) version you're "playing" with ??

It does seem a little on the small side, maybe you could check it with Mediainfo (which can be found in the Ripbot Tools folder, or you can download & install it, very handy).

How big is the original ??

Yes, I know what it's like to have TrueHD issues, I convert mine to DTSHD-MA (bit of a process).

So how long did the job take, finally ??

So here's where you change the Decoder settings (in the latest build of RB) :-

https://www.mediafire.com/view/kxl01...coder.jpg/file
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 28th December 2019, 04:44   #18115  |  Link
stryker412
Registered User
 
Join Date: Feb 2002
Posts: 151
Quote:
Originally Posted by Pauly Dunne View Post
So is that 1080p or 2160p (4K) version you're "playing" with ??

It does seem a little on the small side, maybe you could check it with Mediainfo (which can be found in the Ripbot Tools folder, or you can download & install it, very handy).

How big is the original ??

Yes, I know what it's like to have TrueHD issues, I convert mine to DTSHD-MA (bit of a process).

So how long did the job take, finally ??

So here's where you change the Decoder settings (in the latest build of RB) :-

https://www.mediafire.com/view/kxl01...coder.jpg/file
Sorry this was 4K resolution. The original is 36GB.

How can I get version 1.25? I still have 1.24 which is why I wasn't seeing those options. All the links on videohelp and the first post here are still for 1.24.
stryker412 is offline   Reply With Quote
Old 28th December 2019, 05:17   #18116  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by stryker412 View Post
Sorry this was 4K resolution. The original is 36GB.

How can I get version 1.25? I still have 1.24 which is why I wasn't seeing those options. All the links on videohelp and the first post here are still for 1.24.
OK, well that's sort of strange it's only 4.8Gb.

You didn't resize it, did you ?? Is it still 4K res., 3840 x 2160.

So how long did it take ??

If you're still on 1.24.0, you must not have Auto-update enabled, enable it, (refer to screenshot in previous post) then close RB, then re start it, if you're connected online it will download all the updates it needs (yes Auto-update is working fine again).

Wait a couple of minutes, then check the "update_log.txt", or in the "Updates" folder, to check if there is any files in there.

Then all you need to do is close RB down again, re start, it will automatically update itself, and there you have it.
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)

Last edited by Pauly Dunne; 28th December 2019 at 05:22.
Pauly Dunne is offline   Reply With Quote
Old 28th December 2019, 17:15   #18117  |  Link
stryker412
Registered User
 
Join Date: Feb 2002
Posts: 151
No it was not resized.
stryker412 is offline   Reply With Quote
Old 28th December 2019, 17:27   #18118  |  Link
stryker412
Registered User
 
Join Date: Feb 2002
Posts: 151
Quote:
Originally Posted by Atak_Snajpera View Post
GPU decoding may give you few percent speed boost...
I'm actually getting the opposite. On GPU, I'm averaging 4-5FPS and on CPU 7-8.
stryker412 is offline   Reply With Quote
Old 28th December 2019, 21:28   #18119  |  Link
GZZ
Registered User
 
Join Date: Jan 2002
Posts: 581
Might have found the issue with the info.txt missing.

If I use batch mode and have a custom filter with a UNC path to the tools folder (in this case for MDegrain) in my shared folder, then the script fail if the tools folder is missing, but Ripbot fail to see this and just continue without an error.

I did a video (Screen capture). Please have a look at it: http://gofile.me/6GaFo/v7OO0kwJ

If GetInfo.avs is runned multiple times, then the file (info.txt) is not overwritten, but data is appended to the file.

Last edited by GZZ; 28th December 2019 at 21:53.
GZZ is offline   Reply With Quote
Old 28th December 2019, 21:56   #18120  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Why do you use network paths in Avs?
Atak_Snajpera is offline   Reply With Quote
Reply

Tags
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.