Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th March 2021, 07:38   #19041  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Originally Posted by jlcpremier View Post
I am using distributed encoding and recently upgraded to v1.26 on all machines. I've noticed that on just about every job, a slave machine or two hangs when it finishes encoding that chunk. It will stop at either :02, :01 or :00 seconds left. If I manually disable that node by clicking the network button, it will re-encode that chunk on another node, or I can go into the temp folder and usually the chunk is there so then I just have to run the CombineAllChunks.cmd
We are running a full 16 server distributed encoding farm. I've noticed this problem mainly when using the avisynth filters for Mdegrain2 and KNLMeansCL. Each server is an HP EliteDesk 800G1 w/i7 and 8GB RAM on SSD, solely being used with CPU encoding (no GPU)

Is there any settings I can check or change to make this more reliable? You can see an example of how a job looks when it hangs below
Next time when you get this problem please also show me what encoding server reports in main TAB and in tcp communication TAB. IT looks like string containing "encoded ..." Was not sent back to encoding client.

Last edited by Atak_Snajpera; 26th March 2021 at 07:42.
Atak_Snajpera is offline   Reply With Quote
Old 26th March 2021, 09:05   #19042  |  Link
jlcpremier
Registered User
 
Join Date: Mar 2021
Posts: 3
Quote:
Originally Posted by Pauly Dunne View Post
Welcome to the Forum.

Wow, 16 servers...nice

So was it working OK before the update to 1.26.0 ??

OK, for KNLMeanCL processing you need an appropriate, supported GPU.

MDegrain filters are pretty CPU intense, but shouldn't really cause any problems.

Also, do you know what version of Avisynth you're running (bottom of Encoding Server window is an easy way to check, you can simply run that from the RipBot264 installation folder).

So just out of interest, what were you encoding here (in reference to the screenshot) ?
Yeah working in IT has it's perks

It wasn't getting hung as much but still would occasionally on jobs. I haven't really played around much with the NR filters before, so I think it may be more tied to that. I also notice the nodes take a lot longer to fire off when the filters are in use.

I looked up some info on KNLMeansCL and there wasn't a whole lot of info about it, in terms of ripbot, but in the test file I ran just using KNLMeansCL at levels 1, 3, and 5 for noise reduction it made a substantial difference with the noise reduction, and file size even though there's no real graphics card for it to use the GPU for. Just running it through the Degrain filter at each level didn't actually make much difference in the PQ or the size reduction of the file. For some reason putting them together did though.

Version of avisynth+ shows as 3.4 (r925)

I'm taking all 9 seasons of 24 I have on bluray and trying to compress them down and improve the picture. This particular show has so much noise and grain in the source files that a standard 1080P episode with no noise reduction is still around 2-3GB encoded at slow CQ24, with the Mdegrain2/KNLMeansCL filters set to max it drops down to about 400MB and still looks great. But that's over 200 episodes, so you can see why it being automated would be nice.

Last edited by jlcpremier; 26th March 2021 at 09:14.
jlcpremier is offline   Reply With Quote
Old 26th March 2021, 10:37   #19043  |  Link
slalom
Registered User
 
slalom's Avatar
 
Join Date: Jan 2010
Posts: 456
Quote:
Originally Posted by ReinerSchweinlin View Post
Thanx I know - It can be done manualy. Ma thought was to solve it in Ripbot. Of course one can allways call up any muxer, ffmpeg, any other muxing GUI, etc...
The answer is, you can't
__________________
E5 2697 v2 @ 3.0GHz on P9X79 Deluxe 24GB
Xeon E5-2680 v2 @ 3.1GHz 16GB
Sony Vaio VPC-F13Z1E/B
slalom is offline   Reply With Quote
Old 26th March 2021, 11:13   #19044  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by jlcpremier View Post
Yeah working in IT has it's perks

It wasn't getting hung as much but still would occasionally on jobs. I haven't really played around much with the NR filters before, so I think it may be more tied to that. I also notice the nodes take a lot longer to fire off when the filters are in use.

I looked up some info on KNLMeansCL and there wasn't a whole lot of info about it, in terms of ripbot, but in the test file I ran just using KNLMeansCL at levels 1, 3, and 5 for noise reduction it made a substantial difference with the noise reduction, and file size even though there's no real graphics card for it to use the GPU for. Just running it through the Degrain filter at each level didn't actually make much difference in the PQ or the size reduction of the file. For some reason putting them together did though.

Version of avisynth+ shows as 3.4 (r925)

I'm taking all 9 seasons of 24 I have on bluray and trying to compress them down and improve the picture. This particular show has so much noise and grain in the source files that a standard 1080P episode with no noise reduction is still around 2-3GB encoded at slow CQ24, with the Mdegrain2/KNLMeansCL filters set to max it drops down to about 400MB and still looks great. But that's over 200 episodes, so you can see why it being automated would be nice.
Hi, so have you tried MDegrain3 (or MDegrain4, custom filter you'd have to add yourself)..I find they do an excellent job of cleaning up things.

That's a fairly old version of Avisynth...

I'm also curious about the power consumption with 16 SFF servers running....just looking at your screenshot, even with 16 DE server's your processing speeds aren't that impressive, but that's probably due to the filters you're using.

Just out of interest, under the "Main" tab of RB, do you have "Limit to following filters only" checked ??

Also, under the "Distributed Encoding" tab, you could maybe bump up the "Chunks Size" to 2.

Well, that's my 10 cents worth. It's ALL about speed !!!!

Cheers
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 28th March 2021, 07:50   #19045  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
HQDN3D Revisited...

So, I'm sure some of you remember and used to use this denoising filter when it was an integral function of RipBot264, but was replaced by MDegrain.

It did a pretty good job, I used to use quite a bit (back in the day), and I have only just now decided to figure out how "put it back" with several filtering options (settings acquired off the "Net", and referencing how it worked in a much older version of RB).

So if anyone is interested, you can get it here :-

Link has been removed due to limited feedback, and that the filter isn't "up to scratch".
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)

Last edited by Pauly Dunne; 8th April 2021 at 01:38.
Pauly Dunne is offline   Reply With Quote
Old 28th March 2021, 21:42   #19046  |  Link
slalom
Registered User
 
slalom's Avatar
 
Join Date: Jan 2010
Posts: 456
I get an error, are you sure the file is ok?
__________________
E5 2697 v2 @ 3.0GHz on P9X79 Deluxe 24GB
Xeon E5-2680 v2 @ 3.1GHz 16GB
Sony Vaio VPC-F13Z1E/B
slalom is offline   Reply With Quote
Old 29th March 2021, 12:30   #19047  |  Link
slalom
Registered User
 
slalom's Avatar
 
Join Date: Jan 2010
Posts: 456
WinRar v4.2 gives me error,
Code:
The archive is unknown format or damaged
v5.9 opens it fine
__________________
E5 2697 v2 @ 3.0GHz on P9X79 Deluxe 24GB
Xeon E5-2680 v2 @ 3.1GHz 16GB
Sony Vaio VPC-F13Z1E/B
slalom is offline   Reply With Quote
Old 29th March 2021, 19:51   #19048  |  Link
slalom
Registered User
 
slalom's Avatar
 
Join Date: Jan 2010
Posts: 456
I later installed the v6.0 to be safe

will test that filter some time later
__________________
E5 2697 v2 @ 3.0GHz on P9X79 Deluxe 24GB
Xeon E5-2680 v2 @ 3.1GHz 16GB
Sony Vaio VPC-F13Z1E/B
slalom is offline   Reply With Quote
Old 29th March 2021, 20:52   #19049  |  Link
Ryushin
Registered User
 
Ryushin's Avatar
 
Join Date: Mar 2011
Posts: 431
Quote:
Originally Posted by Pauly Dunne View Post
So, I'm sure some of you remember and used to use this denoising filter when it was an integral function of RipBot264, but was replaced by MDegrain.
I believe HQDN3D was taken out because it did not support 10 bit color. Since I convert everything to 10bit color and x265 (even from 8 bit sources) I'm afraid I can't use it. I also stopped using it because MDegrain2 was such a drastic improvement I never went back. Even when originally I was only getting 2fps from a Blue-ray source when using MDegrain2, the quality was worth the wait.
Ryushin is offline   Reply With Quote
Old 30th March 2021, 00:07   #19050  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by Ryushin View Post
I believe HQDN3D was taken out because it did not support 10 bit color. Since I convert everything to 10bit color and x265 (even from 8 bit sources) I'm afraid I can't use it. I also stopped using it because MDegrain2 was such a drastic improvement I never went back. Even when originally I was only getting 2fps from a Blue-ray source when using MDegrain2, the quality was worth the wait.
Hi, like I said, I haven't tested it with any x265 encodes, yet.

I still do a lot of x264 encodes.

Interesting that this filter is still used in Handbrake, and Staxrip.

I think why I did this is, I used to use it a lot, it did a pretty good job, and is a lot faster than MDegrain, and its just another denoising option that someone might find a use for, that got "axed" !!!
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)
Pauly Dunne is offline   Reply With Quote
Old 30th March 2021, 13:30   #19051  |  Link
slalom
Registered User
 
slalom's Avatar
 
Join Date: Jan 2010
Posts: 456
Quote:
Originally Posted by Ryushin View Post
I also stopped using it because MDegrain2 was such a drastic improvement I never went back.
I will have to agree

I just checked the preview for 3 movies, HQDN3D_strong vs MDegrain1

MDegrain1 is way better for my eyes. I don't know about encoding speed
__________________
E5 2697 v2 @ 3.0GHz on P9X79 Deluxe 24GB
Xeon E5-2680 v2 @ 3.1GHz 16GB
Sony Vaio VPC-F13Z1E/B
slalom is offline   Reply With Quote
Old 30th March 2021, 17:12   #19052  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
Quote:
Originally Posted by slalom View Post
I will have to agree

I just checked the preview for 3 movies, HQDN3D_strong vs MDegrain1

MDegrain1 is way better for my eyes. I don't know about encoding speed
Same here, MDegrain is much more natural and transparent looking for me.
ReinerSchweinlin is offline   Reply With Quote
Old 31st March 2021, 13:42   #19053  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
Quote:
Originally Posted by Pauly Dunne View Post
And as a side note, Handbrake only seems to have 2 denoiser's, HQND3D & NLMeans, so they don't know what they're missing
Yes, Handbrake is very stable and user friendly and doesn´t change too often - one of the reasons why many Benchmarks use it and users like it. But it misses some very basic and advanced features, which others like Ripbot, Staxrip, etc... offer. Like:

- Very high Quality deinterlacing
- GPU Denoising
- Normalizing Audio
.
.
.

On the other hand, its very good documented, stable, many nice features for batch encoding, cross-plattform, etc...

Each to his own...

Last edited by ReinerSchweinlin; 7th April 2021 at 07:24.
ReinerSchweinlin is offline   Reply With Quote
Old 31st March 2021, 22:55   #19054  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
High quality deinterlacing is also absent in handbreak.
Atak_Snajpera is offline   Reply With Quote
Old 5th April 2021, 08:46   #19055  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
More filter options...

So I have been doing some testing with different Grain & Denoising filters, some that are not part of Ripbot264.

However, they can be found in Handbrake, Staxrip, & Hybrid.

One thing that this has shown me, is that HQN3D just isn't really "in the race".

So the original footage is VERY grainy / noisy, and even though MDegrain 2 & 4 did a pretty good job, in a reasonable time, I found that the SMDegrain grainy did the best job, however, to get those results, it takes a L-O-N-G time, as an example of that the SMDegrain grainy took a bit over an hour to process a 4:40 sample, whereas MDegrain 4 only took about 23 minutes.

TemporalDegrain2 took nearly 1:40:00

TNLmeans took just over 00:47:30

xNLmeans took 2:16:00

I just want to add that these times were on an old slightly overclocked Intel i7 3930K, using DE, but only 2, 2minute chunks.

So even thought these filters do a very similar job, there is a big difference in encoding times, versus the end result, fortunately (or not) at least RipBot has the massive advantage of DE, I would hate to see how long the other popular encoders would take.
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)

Last edited by Pauly Dunne; 8th April 2021 at 01:26.
Pauly Dunne is offline   Reply With Quote
Old 5th April 2021, 16:53   #19056  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Originally Posted by Pauly Dunne View Post
How about English !!!!!

I've just noticed that you left 25 of these crappy posts on many of this Forums topics...go somewhere else !!!!

Online Casino bullshit !!!
You are yelling at the bot...
Atak_Snajpera is offline   Reply With Quote
Old 6th April 2021, 07:31   #19057  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,752
Better report such posts to the moderators instead.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 7th April 2021, 07:29   #19058  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
Issue with multiple audio tracks:

Hi all... I remember reporting this a while ago and just noticed the behaviours still being the same:

- when I do a batch encode and tell RB to just copy audio - only the first audio track is copied. Same behaviour when audio is encoded.

Anybody else having this issue?
ReinerSchweinlin is offline   Reply With Quote
Old 7th April 2021, 10:37   #19059  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 692
Quote:
Originally Posted by ReinerSchweinlin View Post
Issue with multiple audio tracks:

Hi all... I remember reporting this a while ago and just noticed the behaviours still being the same:

- when I do a batch encode and tell RB to just copy audio - only the first audio track is copied. Same behaviour when audio is encoded.

Anybody else having this issue?
Well, I don't use Batch all that much, but I just loaded a couple of files, using batch, and unless I'm blind, I cannot see ANY option for multiple audio tracks, however, you could add up to another 2 tracks per job, once the batch process has created the job(s).

I was going to show a screen shot, but there's no point

However, having said that, for those who use batch a lot, it would be a very handy option to have....
__________________
Not poorly done, just doin' it my way !!!
Live every day like it's your last, because one day, it will be !! (M$B)

Last edited by Pauly Dunne; 8th April 2021 at 01:44.
Pauly Dunne is offline   Reply With Quote
Old 7th April 2021, 21:17   #19060  |  Link
slalom
Registered User
 
slalom's Avatar
 
Join Date: Jan 2010
Posts: 456
Quote:
Originally Posted by ReinerSchweinlin View Post
Issue with multiple audio tracks:

Hi all... I remember reporting this a while ago and just noticed the behaviours still being the same:

- when I do a batch encode and tell RB to just copy audio - only the first audio track is copied. Same behaviour when audio is encoded.

Anybody else having this issue?
Yes, obviously you have to add it manually
__________________
E5 2697 v2 @ 3.0GHz on P9X79 Deluxe 24GB
Xeon E5-2680 v2 @ 3.1GHz 16GB
Sony Vaio VPC-F13Z1E/B
slalom is offline   Reply With Quote
Reply

Tags
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.