Convolution3D

[int 21h]

http://www.vidomi.com/docs.php?myfaq=yes&id_cat=13&categories=Encoder+-+Network+Options

This does encode 'blocks' of frames on the slave machines...

[meleth]

As i understand it, that one just makes 1job/cd and then encodes that job on a diffrent computer.

[int 21h]

Yea, 'faux' distributed encoding :p

[meleth]

When i talked about blocks of frames i meant more along the lines of 1000frames or however the user configures it.

The conclusion i've reached is that if this was to get implemented it would have to get done in both the codec (as you pointed out) but also in the encoder program. Atleast imo it would be best to keep as much as possible of the code in the encoder to make it easier for codec developers.

I just thought of one thing that should be doable without any changes to a codec though. When you do 2pass encoding you should be able to do them at almost the same time via a distributed solution.

[Acaila]

Actually 2-pass mode is the least efficient one if you want to do that over a network. This is because the first pass has to be done on a single computer, leaving you with a minimum encoding time of 50% compared to a normal encoding (if the 2nd pass were to be completed instantly through a network).
Then you have the problem if you give each computer in the network an equally large piece it will slow down encoding to the speed of the slowest computer. Unless you make a test first to determine the encoding speed of each computer and base the block size on this, which in turn greatly complicates writing a program that can do that.

This whole distributed-encoding-thing has been discussed many times before, it's just too much of a hassle with too few benefits. Even if implemented correctly it will not save you a lot of time on an encode, and the amount of things that can be implemented wrongly is simply too great (if you still want to keep optimum quality that is, which Vidomi does not do).

SMP encoding on the other hand can be much more rewarding (because you don't have to break the movie up into several pieces, and thus keep optimum quality), but this is also quite hard to code.

[int 21h]

If you went to a lower level in the codec, you could run whatever pass or mode you wanted to distributively, because, ideally, the distributed nature would be linked to raw calculations, and not frames or what not. (Similar to the currently in use distributed projects by Intel (i.e. cancer research, genes, anthrax resesarch, etc))

Even SMP isn't a silver bullet, because most people are only on two processors, so you've already got two very clear cut processes going on. Decompression, Compression.

[WarpEnterprises]

Back to Convolution3D:

I notice a red hue after using C3D - anybody else affected?

(Applying it 10x and I see it VERY easy...)

[MaTTeR]

Actually in my latest C3D tests I'm seeing a green hue in parts of the images but then they disappear. I don't know for sure if this is for sure an issue with C3D or possibly ffdshow though. I need to investigate further.

Edit- Sorry, false alarm from my side. The nasty green hue I'm seeing has been narrowed down to an ffdshow problem.

[manono]

Yeah Warp-

I've noticed the slight red hue also-I used C3D twice once and it was a bit more noticeable. Faces sometimes seem a bit flushed.

[sh0dan]

Yes - C3D has a rounding bug - use compare() to see the hard numbers.

[vlad59]

Oooops. I thought I've fixed most of them.

I already fixed (it's not released yet) a rounding bug on the temporal blur done on first and last line. But there could be other bugs.

@ShOdan
Can you explain me how do you check that with compare ?

Thanks advance.

EDIT : fixed some typos

[sh0dan]

@vlad: This was the script I used:


avisource("d:\video\leo\leo-2d-samlet-2.avi")
convertbacktoyuy2()
c3d=Convolution3d(0,8,8,8,8,3,0)
c3d=Convolution3d(c3d,0,8,8,8,8,3,0)
c3d=Convolution3d(c3d,0,8,8,8,8,3,0)
c3d=Convolution3d(c3d,0,8,8,8,8,3,0)
compare(c3d)


Mean abs dev. (Mean absolute deviation) shows the average difference between the original and your filter - not of much use, when your filter changes the image (as yours). It does X = sum_of_all( abs(pixel - org_pixel)) / pixels


Mean dev. Shows the difference WITHOUT using an absolute value. This shows if there is a bias toward rounding up/down - if rounding is correct, this value will be close to 0, since statistically half the pixels are rounded down, and the other half is rounded up.
It does X = sum_of_all(pixel - org_pixel) / pixels - (can't remember if pixel and org_pixel have to be switched).

[vlad59]

Yep

Got it !!
Already found it : this rounding bug only exists with matrix 0 and it's due to multiple pavgb use.

It will be release tomorrow. I need some time to test it before.

Thanks a lot for this script sh0dan

[Defiler]

Don't forget to hook me up with a copy so that I can host it. :)

[vlad59]

@Defiler

Don't worry, you'll receive a mail. ;)

@all
I'll be a little late. Next version of Convolution3d will be released tonight.

Yesterday I thought I could have time to fix the rounding bug (it's done) and to finalize the small improvements I began (done but not enought testing). But to be sure to have no (too much) bugs. I'll make two releases.

Sorry again

[vlad59]

I'm again a little late. but I fixed all the rounding bugs of convolution3d. So the beta4 is only a bugfix release, you'll have to wait for the 1.0 to have new functionality.

The beta4 will be available at http://hellninjacommando.com/con3d/ as soon as Defiler has time to update the page.

I've also attached the beta4 to this post.

[Defiler]

OK. The site is updated, at the URL that Vlad just mentioned.

[iago]

@vlad59,

[...] for normal film : Convolution3d (0, 8, 8, 8, 8, 3, 0) or
Convolution3d (1, 8, 8, 8, 8, 3, 0) are way better if you don't want to huge blur on characters faces. [...]
from the readme file

Imho it would be better to update these lines in the readme file as well, since (0/1, 8, 8, 8, 8, 3, 0) also seems to go too harsh and sweep away too many details (together with the noise) for normal and clean films.

According to the latest tests, perhaps it's more proper to say:
values such as (0/1,4,4,4,4,2.8,0) can be more suitable for normal and clean movies.

kindest regards and thanks for the new release,
iago

[vlad59]

@iago

Ooooops, you're right iago. I followed all your test and agreed on your conclusion.

This will be fixed in next release.

Thanks for reporting ;)

Regards

[MaTTeR]

Think I would have to go along with that also guys. In fact (0,4,4,4,4,2.6,0) even seems to be a bit harsh on my eyes for some movies. I'm still struggling trying to find a setting I like on clean progressive DVD sources but it just seems to blur details a bit much. Guess it's more suitable for "dirty" sources or anime in general.

[Defiler]

vlad: I'd be happy to update the readme on the website, if you can tell me which settings are now recommended for various types of sources.

[vlad59]

Ooops haven't seen your post Defiler. I'll update my readme and mail you today.

Thanks

[unplugged]

I have made some (comparison) tests using beta4 Convolution3d and it seems to me that Matrix 1 is somewhat more precise than 0 with temporal jobs like Convolution3d(1,0,0,4,4,3,0).
What I point is that matrix 1 isn't so worst than 0 for low-noise content, with temporal intervention matrix 1 can be more precise than 0 (I think due to nature of its matrix coefficients), of course with lower thresholds.

I have considered many times a specific C3D setup oriented for static noise removal (with 0,0 spatial thresholds) by finding good temporal values, there are several movies with noise and lot of detail at same time, these are hard cases and need perfect compromise! ;)

[vlad59]

In fact, I never thought of using convolution3d by keeping only Temporal thresholds ;).

It's normal that matrix 0 produce worse results with your parameters :
If you use 0, 0 as spacial thresholds then
Current Frame weight = ComputedPixel * 32
And NewPixelValue = PreviousFrameWeight + NextFrameWeight + ComputedPixel * 32 / 64

So in this case the original value of the pixel has a lot of influence.

With Matrix 1 :
Current Frame weight = ComputedPixel * 9
And NewPixelValue = PreviousFrameWeight + NextFrameWeight + ComputedPixel * 9 / 27

So here the original pixel has less influence.

If my explaination are not clear enought (in about 95% of time ;)), you can have a look at the beginning of the thread named Smooth in time and space. I've explained with an example how work Convolution3d.

If you want I can build a test version with no use at all of the current frame to use pure temporal informations.

[theReal]

The last movie I encoded using convolution3d was Cruel Intensions (PAL, region 2) - I only wanted to improve the looks because the compressibility of this movie is really good.

For this special case I found that using a relatively strong setting (1,6,8,6,8,2.8,0) together with a little sharper resizing (bicubicresize(640,352,0,0.6)) gave very good results. The details were kept, but the movie's annoying artefacts were filtered away.

Another movie that I could improve with convolution3d recently was Sixth Sense (also PAL, region 2). This movie also shows annoying artefacts even when encoded with quant 2. Convolution3d(1,3,5,2,4,2.8,0) helped a lot and made the result look better without blurring too much.

I think the best settings depend a lot on how the source movie has been pre-filtered, edge-enhanced (or not) and so on.
What I think is great is that C3D is the first filter that I can use to visually improve even quant 2 encodings.
Thanks again for that great tool!

[bb]

Originally posted by vlad59
In fact, I never thought of using convolution3d by keeping only Temporal thresholds ;).
That's exactly what I thought when I read unplugged's post. For this purpose you'd better use a dedicated temporal filter (should be faster).

bb

[vlad59]

@TheReal

Yep that's exactly the way I'm using Convolution3d. As I mainly encode/test animes I use the following script :
Convolution3d (1, 8, 16, 8, 8, 2.5, 0)
LanczosResize (XXX, XXX)
MSharpen () -- if necessary

With I don't gain a lot in compressibility (roughtly 5 - 15% depending of the strength of C3D), but the result is often good (I mean clean and sharp enought at least for my taste ;)). I have to test a little more warpsharp to see if it can help.

I have never encoded "the Sixth Sense". I own this DVD, so I'll make a test just for curiosity.

@bb and unplugged

I remember a thread some weeks ago about using only temporal data if motion is small. I think Siwalters made a test filter and it wasn't very effective.

But as always I may be totally wrong :( . So please excuse me in advance if so :rolleyes:

[unplugged]

thanks, really I hadn't tried TemporalSoften or TemporalSmoother ... for some stupid reason I have fastly speculated that the last arrived (C3D) was less worst of the group... sorry for previous post

Now, by trying in this right moment TemporalSoften (AVS 2.06) I believe to have found a very good balance point with parameters: it satisfies me a lot.
What interest to me firstly is to remove noise fluctuation and secondly to grab the (little) bitrate gain; I don't like blur minimally 2D detail not even if it mainly applies to motion (but this is not the case of anime content, of course animes usually need heavy 3D smoothing).

Noise firstly irritates me :), apart the bitrate eat problem it brings the encoder to generate even more annoying macroblock fluctuations.
Plus, in these cases with XviD I always use H.263 q. matrix, with not clean content MPEG tends to emphasize artifacts...


[Edit]
P.S.: Any experience about results/diff. between TemporalSoften and TemporalSmoother?
When someone talks about TemporalSmoother, is it intended the one contained in MPEG2DEC.DLL ?

[SansGrip]

unplugged: Now, by trying in this right moment TemporalSoften (AVS 2.06) I believe to have found a very good balance point with parameters: it satisfies me a lot.

For noisy sources (e.g. captures from VHS) I use a combination of Cnr2(), TemporalSoften(x, x, 0) and MSharpen(x, x) before Lanczos resizing to 352x240. I've found this gives me the best compromise between reducing noise and maintaining detail and results in a fairly sharp image.

unplugged: P.S.: Any experience about results/diff. between TemporalSoften and TemporalSmoother?

In my experience TemporalSoften() gives comparable results and is slightly faster, but I could be missing something.

unplugged: When someone talks about TemporalSmoother, is it intended the one contained in MPEG2DEC.DLL ?

That's the only one I know of.

[Piper]

First, thank you all for providing these filters...this is really great stuff.

@vlad59

>Yep that's exactly the way I'm using Convolution3d. As I mainly encode/test animes I use the following script :
Convolution3d (1, 8, 16, 8, 8, 2.5, 0)
LanczosResize (XXX, XXX)
MSharpen () -- if necessary

Thinking aloud....

That's precisely the same method I've started using with my encodes, though I'm still fiddling around with my settings (aren't we all?).

For tv caps, I'm currently using:
Convolution3d(0,4,4,4,4,2.8,0)
LanczosResize(384,288) <--From 720x480
MSharpen(threshold=15,strength=50)

C3D gives good results and is faster than combining 2D Cleaner and Temporal Smoother filters. The trailing effect from the Temporal filter (even with very light settings) seems to be a thing of the past. I don't pretend to understand the the C3D settings however. These settings are used from piecing together the results from other posts. If there are other Howto's on the subject beyond the .txt provided with the fitler....post please?

I'm having very good success with LanczosResize and I particularly like it's tendancy to sharpen the image, esp since DivX tends to have the opposite effect. :-)

I've only started to use MSharpen and so far I like what I see. There's some fine tuning to be done with the threshold, but I'll get there. Up until now, I've been using UnSharp Mask but have been dissapointed with it's tendancy (combined with BiCubic resize) to emphasize (if not create) halo's (aliasing?) around sharp edges.

Again, thanks for the filters, my encodes keep getting better and better as a result.

[jarthel]

body=mpeg2source("d:\dvdrips\lodoss14\lodoss14.d2v").trim(0,40176).Telecide(Guide=1,Gthresh=40,Threshold=20,Chroma=True,dthreshold=10,Post=true).Decimate(cycle=5).crop(14,4,692,472).temporalsmoother(5).Convolution3d (0, 8, 16, 8, 8, 3, 0).lanczosresize(640,480)

return body
-----------------

Tested that script last night on Record of Lodoss War. Very messy anime, lots of interlacing and quite noisy. Results are quite good and comparable to SSHiQ. And much much faster than SSHiQ! It save around 30 minutes/pass. Thank you Vlad.

jayel

[bb]

Originally posted by jarthel
body=mpeg2source("d:\dvdrips\lodoss14\lodoss14.d2v").trim(0,40176).Telecide(Guide=1,Gthresh=40,Threshold=20,Chroma=True,dthreshold=10,Post=true).Decimate(cycle=5).crop(14,4,692,472).temporalsmoother(5).Convolution3d (0, 8, 16, 8, 8, 3, 0).lanczosresize(640,480)

return body

What if you skip the temporalsmoother?

bb

[JohnMK]

Can you guys recommend a decent commandline for The Simpsons, NTSC? :)

[HarryM]

@vlad59:

I have special question at Convolution3D_YV12.

You present, that Convolution3D works at matrix 3x3x3 pixels.

Luma filtering is 3x3x3 pixels (logicaly).

But what chroma filtering? At YV12 is 1 logical chroma pixel = 2x2 physical pixels, right?

Chroma filtering is
a) 3x3x3 physical pixels too or
b) 3x3x3 logical pixels (6x6x3 physical pixels)???

When it a), can you make on b) for speed-up filtering?

[theReal]

What if you skip the temporalsmoother?

For very noisy sources, temporalsmoother before C3D is quite good. I've done that recently on a tv-capture and the result was great. In my experience, C3D alone is not perfect for removal of very strong noise - temporalsmoother's smoothing effect is much stronger (which can be good or bad, depending on the source quality)

I've repeatedly said I very much like C3D to encode mediocre DVDs to 'perfect' Divx - but for my Simpsons tv captures I'm still using this filter chain (avisynth 2.06):

temporalsmoother(4,1)
spatialsoften(2,5,7)
temporalsmoother(4)

This is much too strong for anything else than cartoons, but for exactly that it's more effective than C3D.

[Shayne]

Been playing with this filter for some time now and i am getting fantasic results on my 720x480 mpeg2 tv caps with

Convolution3D (0, 10, 17, 10, 13, 2.8, 0)
unfilter(+#,+#)

[Wario]

Originally posted by theReal
For very noisy sources, temporalsmoother before C3D is quite good. I've done that recently on a tv-capture and the result was great. In my experience, C3D alone is not perfect for removal of very strong noise - temporalsmoother's smoothing effect is much stronger (which can be good or bad, depending on the source quality)

I've repeatedly said I very much like C3D to encode mediocre DVDs to 'perfect' Divx - but for my Simpsons tv captures I'm still using this filter chain (avisynth 2.06):

temporalsmoother(4,1)
spatialsoften(2,5,7)
temporalsmoother(4)

This is much too strong for anything else than cartoons, but for exactly that it's more effective than C3D.

Agree with TheReal because my Capture video (music clip) have NOISE and Convolution3d remove the noise but not perfectly, TemporalSoften remove more...

[jarthel]

Originally posted by bb
What if you skip the temporalsmoother?

bb

sorry for the late reply.

It's not as good and the I've realized since that temporalsmoother(5) makes a video more compressible.

jayel

[Holomatrix]

sorry to bud in, but would you recommend this 3d filter with low bitrate DVD encodes? or just stick with TemporalSmoother? I'm doing some test's encodeing DVD to a 900 bitrate just to see what are the best filters do yeid acceptable quality with no macroblocks. Thanks

[serbersan]

@vlad59

What about a new (another) preset for credits treatment??? I mean a very agressive one.

Not only, HQanime, LQmovie... besides for credits.

Thanks.

[Didée]

Holomatrix,

when aiming for low bitrate encodings, I would definetly give Convolution3D a try. You can achieve much better results than with temporalsmoother alone - though you might have to wait a little longer for the C3D encoding to finish ...

For the macroblocks -
the more compressable you make the source by smoothing and/or removing details, the higher the risk is that you get macroblocks. Since some areas may become too "flat" by the smoothing filters, the codec might decide "to pull out its biggest hammer" on these areas - that's more or less by mpeg-4's design.
Perhaps you might be interested in this thread (http://forum.doom9.org/showthread.php?s=&threadid=37135). The discussion there is about fighting mpeg macroblocks and - are you sitting firmly - adding noise to the source.

Just for curiosity, a few days ago I tried to put LOTR extended version (~ 3h 20min) on 1 CD, pulling out most of my filtering knowledge. It turned out that the result looks *astonishingly* good on my 34" TV-set (but for viewing on CRT, of course I prefer my 2CD backup).

Remark: I OWN this DVD box ;)

[Holomatrix]

Ok, thanks for the info. Do you mind posting the script you used? and I'll read that thread.
Thanks

[vlad59]

@Holomatrix

I don't really recommend anything as I don't have enought time to encode a lot :( .

But I created Convolution3D exactly for this problem a very noisy low quality anime on DVD.
With Convolution3D you don't remove all the noise if the source is really noisy (other filters are way better for this : PeachSmoother for example) but you soften enought the source to remove most of combing artefacts and source macroblocks and you make your source more compressible.
With anime source I really want to smooth a lot you can use msoften in addition (it really does a good job).

Later, thanks to iago, we found that it helps compressibility even on hi quality movie source without losing too much details.

@serbersan

I plan to make another YUV2 version of Convolution3D soon to synch with the alpha version I did for Avisynth 2.5.

I'll add that.

[iago]

Originally posted by vlad59
[...]Later, thanks to iago, we found that it helps compressibility even on hi quality movie source with losing too much details.[...]@All Convolution3D users!

Do not panic! Despite the fact that Vlad, the author of this great filter, tries to misinform and mislead you from time to time, though I don't know why, Convolution3D is absolutely one of the best in keeping details! :D

With his above claim, Vlad is absolutely sabotaging this amazing filter; don't believe his lies! Believe me! :D

The truth is as below:

[...]Later, thanks to iago, we found that it helps compressibility even on hi quality movie source without losing too much details.[...]

regards ;),
iago

[vlad59]

@iago

Ooooops, I really should reread my posts more often.
Thank you very much for fixing this big typo ;) ;) ;)

I've edited the previous post .... :rolleyes:

It's a really good thing you was here iago.

Take care.

[iago]

Vlad,

Thanks are for you, man, for your outstanding filter! ;) Keep up your great work! ;)

best regards,
iago

[mikeson]

I just cannot stand away when there is so much flattering here ;)

Vlad, I have to say that your C3D is so outstanding filter, I use it almost in every script (to be honest, in every :D ), because all DVD I've got on my HDD are noisy (some less, some more, but it is always there).

Keep up good work! :)

[dvd2svcd]

This filter so good that it makes Blair Witch Project look like it was shot with regular studio cameras :D

[Marc FD]

if i could only choose one denoiser, it would be c3d ^^

for MPEG-4 blocking, it'll always look better with c3d than without.
if you still think you get MORE blocking with c3d, add a bit AC noise, it's the best way to add noise on movie encodes. i don't use it, because i only encode anime.

[HarryM]

Yes.

For Avisynth exists many good temporal/spatial filters, but Convolution3D is the best!

Convolution3D is NUMBER ONE for me.


I vote for implementing Convolution3D direct into Avisynth.:D