Convolution3d(0,8,8,8,8,3,0) versus Normal Encode test results

[iago]

FIGHT CLUB - Convolution3d(0,8,8,8,8,3,0) versus Normal Encode test results:

MOVIE: Fight Club (2hr-19min) NTSC 1
AIMED VIDEO SIZE: 644000kb (629mb)

SmartRipper 2.40
DVD2AVI 1.76 (Forced FILM 29.97->23.976)
Avisynth 2.05
VirtualDub 1.4.10

XviD 2pass / Koepi's 18082002-1 build
(quantizers capped 2-6/2-16 in second pass for both encodes, credits 20quant)
(exact same encoding parameters except the ones mentioned here)


1) normal encode (usually preferred routine):
--------------------------------------------------
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
mpeg2source("C:\FIGHT\FIGHT.d2v")
crop(4,60,716,354)
BicubicResize(640,272,0,0.5)
Trim(0,196015)+Trim(196016,0).TemporalSmoother(10)
--------------------------------------------------
(640*272, NeutralBicubic, MPEG/Modulated quantizers, lumi masking in both passes)


2) convolution3d(0,8,8,8,8,3,0)
----------------------------------------------------
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\convolution3d.dll")
mpeg2source("C:\FIGHT\FIGHT.d2v")
crop(4,60,716,354)
BicubicResize(640,272,0,0.75)
convolution3d(0,8,8,8,8,3,0)
Trim(0,196015)+Trim(196016,0).TemporalSmoother(10)
----------------------------------------------------
(640*272, SharpBicubic, MPEG/MPEG quantizers, no lumi masking in both passes)


1stPass Results:
----------------
1) normal encode:
1stPass/2ndPass ratio: 2.12 (1335/629)
GordianKnot compressibility value: 50.1

2) convolution3d(0,8,8,8,8,3,0)
1stPass/2ndPass ratio: 1.86 (1170/629)
GordianKnot compressibility value: 57.1

2ndPass Results:
----------------
1) normal encode:
Average quantizer: ~ 3.35

2) convolution3d(0,8,8,8,8,3,0)
Average quantizer: ~ 3


Comments/Observations:
* There is a considerable difference between the 1stPass/2ndPass ratios (compressibility) of normal encode and convolution3d(0,8,8,8,8,3,0) encode although SharpBicubic resizing, MPEG/MPEG quantizers and no lumi masking is used with convolution3d(0,8,8,8,8,3,0), which is a big plus for the filter.

* No visible artifacts or ghosting effect is observed with convolution3d(0,8,8,8,8,3,0) in "this" test.

* Most blocks in keyframes are eliminated in convolution3d(0,8,8,8,8,3,0) encode. Less blocks besides keeping enough/fine details, which is another plus for the filter.

* Important!: Due to limited number of tests with normal movies using convolution3d(0,8,8,8,8,3,0), the results may be case-specific and differ from movie to movie. Certainly more tests should be done to achieve more standard/general results.

Finally, the screenshots from both encodes are attached below for comparison. The decision is yours. After all, it's a matter of personal choice/taste which way to go ;).

best regards to all,
iago

[vlad59]

Thanks iago,

You are, how could I say that, hummm a serial tester ;) ;) ;)

For normal film you should keep all tresholds equal or below 8 (even if the movie is noisy because the blur effect will help the compressibility but kill the quality)
I think you should stay with matrix 0 (first parameter) and use matrix 1 for analog capture, anime or maybe noisy movie (as always it's a matter of taste : do you prefer blur or blocks ???).

That's my thought after testing with excalibur. I'll make a real second pass with soft bicubic (I don't like too sharp movies) to see this real case.

But an interesting thing is that with movies you can lower the temporal influence to 2 without losing too much compressibility (but you'll have more speed).

I'm waiting for the screenshots

[iago]

@vlad59

Thanks ;), it's a pleasure testing your filter, which did great job in this test imo.

Any opinions after analyzing the screenshots?

best regards,
iago
(LOL, serial tester ;))

[Koepi]

I just took a look at the screenshots, and they resemble what I experienced myself:
For DVD ripping of "normal" movie material, the smoothing is killing some details, you can see it better when playing back fullscreen.

I think it's best for TV rips, but for DVD I'd like to go without filtering, except the source is really bad (which it isn't in the case of The Matrix or Fight Club).

Just my 2 cents,

regards,
Koepi

[vlad59]

@iago
Sorry, I got no time to analyze your screenshots, maybe tomorrow.

@all
Have you tested this :
convolution3d(0,8,8,8,8,-1,0) or
convolution3d(0,4,8,8,8,-1,0)
You'll only have spacial filtering but it work well (more details, only 8% less compressible) and it is faster that normal convolution3d.

I'll continue with Excalibur (as it is very detailed) to find acceptable parameters.

[soulfx]

@iago, thanks for taking the time to run some tests and pull some screenshots. It's got me thinking about convolution3d. Tonight I'll start running some tests of my own to see what settings work the best for me (using your test as a guide).

@vlad59, your last couple of suggested settings sound interesting. I haven't messed with convolution3d much and didn't know only spatial could be used. Thanks for pointing that out.

Peace,
SoulFX

[drizztcanrender]

@koepi said:

I think it's best for TV rips, but for DVD I'd like to go without filtering, except the source is really bad (which it isn't in the case of The Matrix or Fight Club).

I agree.
I ran some tests with red rose and convolution3d(0,8,8,8,8,3,0)
and there was some noticable blurring.But it did wonders for me in a very old dvd.Took away most of the noise and reduced filesize from ~730mb to 560mb!

[kilg0r3]

anybody with a fast machine could test the effects when putting conv3d before the resizer?

[iago]

@kilg0r3

I totally agree with you ;). And I guess it would be better with SharpBicubic resize to prevent over-smoothing.

best regards,
iago

[drizztcanrender]

anybody with a fast machine could test the effects when putting conv3d before the resizer?

I'm currently trying this.I'll post back with the results.

[kilg0r3]

yeehaaah!

[drizztcanrender]

Well i did a first small test.
I used this avs:

LoadPlugin("C:\Documents and Settings\ted\My Documents\dvd-ripping\GordianKnot\MPEG2DEC.dll")
LoadPlugin("C:\Documents and Settings\ted\My Documents\dvd-ripping\GordianKnot\Convolution3D.dll")
mpeg2source("C:\Documents and Settings\ted\My Documents\dvd\forester\test.d2v")
crop(1,73,710,429)
convolution3d(0,8,8,8,8,3,0)
BicubicResize(640,272,0,0.75)

Then resulting image was clearer and sharper,especially in closer areas-than the one with conv3d after the resize.And the speed decrease was not that much(about 1frame in 1pass and 3frames to the 2pass.

Then i did the same test with convolution3d(0,8,8,8,8,-1,0),just to test the speed really :).And it was fast indeed!I had an increase of 8 frames :eek:
Again the same results,before resizing looked better,sharper and less blurry.But still not good...anyone tried (really) lower settings??
I'll make some full-length tests tonight on a more noisy source...

[Acaila]

...anyone tried (really) lower settings??
I'm currently processing TemporalSoften(1,5,0), Convolution3D(0,4,0,8,4,3,0) and Convolution3D(0,4,4,8,8,3,0) on The Matrix. All before resizing.
I'll post back when they're done.

Ps. Welcome to this board Drizzt :)

[vlad59]

@acaila

That's strange I would prefer to lower the luma treshold rather than chroma treshold ?

So I prefer going for :
Convolution3D(0,2,6,2,6,3,0)
than
Convolution3D(0,4,0,8,4,3,0)

I haven't done any tests, it's just theory (I may be totally wrong ;) ).

@drizztcanrender

Yes, spacial filtering is way faster and it should produce ~ the same quality on normal movies.
As you said, you should try to lower a little the tresholds.

[Acaila]

@vlad59:

You are correct. Since changes in luma are more visible, the most sensible way is to smooth chroma more than luma. However from what I've seen XviD's MPEG quantizer matrix creates specifically luma noise, so that's what I want to get rid of. This probably means that the chroma threshold can still be set even higher than the luma threshold, but I'm trying to keep all the smoothing to a minimum, just get rid of the luma noise.

Probably doesn't make sense, but oh well :)

Besides, this is my first shot at using this filter of yours. I want to see how it looks first. Maybe after this I'll try your settings. thanks.

[drizztcanrender]

@vlad59

Yeah i noticed that the result was almost the same.I'll try use lower settings and i'll post back.I remember that with the earlier version i had tried some(stupid? :)) settings starting from (0,3,3,3,3,0) and going to (0,8,8,8,8,0).It was on a clean source again but if i remember correctly the 3,4 looked really nice and from 5 and it started to be a little over blurred.About the filesize decrease it came only with (0,8,8,8,8,0),all the other settings had the same filesize.Anyway,back on testing :)

@Acaila

Long time no see :)
Thanks man,it's nice to be here :)

[iago]

Hello again everybody

I can't believe my eyes ;), great to see so many tests going on now :). I would be very pleased if you also provide information about quantization type and resizing method besides convolution3d settings in your tests. I believe these two parameters also act upon the quality/blurriness of the final encode.

best wishes to all,
iago

[drizztcanrender]

Ok i'm going to test this on Crimson Rivers(what a movie!) and i'm going to use xvid.Since i'm not that experienced i would like to post my settings and maybe someone wiser can point me out some better :)
I'm using Koepi's build 18082002 and i'm thinking of:
UltraHigh - 6 motion search precision , MPEG/MPEG quntizers, lumi masking.Quants are 2-6/2-16 and credits 20.Then it's SharpBicubic for resizing.What do you think?

Edit:type-o

[iago]

@drizztcanrender

LOL! Absolutely it's not me, that wiser guy around ;), but I would go with those settings too, maybe without lumi in both passes (?)...

BTW, are you gonna try convolution3d(0,8,8,8,8,3,0)?

Good luck, waiting for your results,
iago

[manono]

Hi drizz-

Crimson Rivers(what a movie!) I couldn't agree more. I love that movie.
UltraHigh - 6 motion search precision -Always
MPEG/MPEG quntizers-For 2 CDs, always (except anime). For 1 CD, it depends.
lumi masking-Personally, I never use it, but plenty of others do.
Quants are 2-6/2-16-for testing, maybe. But ordinarily I limit the heck out of them. But different people do it differently.
credits 20-I use 10, others use 31-to each his own.
SharpBicubic for resizing-Personally I don't touch it-I much prefer BicubicResize(xxx,xxx,0,0.5), but again, different people do it differently. I certainly wouldn't use it for a 1 CD rip, though.
Since i'm not that experienced ...-Not experienced using XviD? Maybe, but based on your few posts so far, you've learned a lot somewhere.

[iago]

@manono

Originally posted by manono
UltraHigh - 6 motion search precision -Always
MPEG/MPEG quntizers-For 2 CDs, always (except anime). For 1 CD, it depends.
lumi masking-Personally, I never use it, but plenty of others do.
Quants are 2-6/2-16-for testing, maybe. But ordinarily I limit the heck out of them. But different people do it differently.
credits 20-I use 10, others use 31-to each his own.
SharpBicubic for resizing-Personally I don't touch it-I much prefer BicubicResize(xxx,xxx,0,0.5), but again, different people do it differently. I certainly wouldn't use it for a 1 CD rip, though. I agree with you on almost all of the above, but why I try/focus on MPEG/MPEG and SharpBicubic is that, from my previous tests, convolution3d(0,8,8,8,8,3,0) already increases compressibility considerably, and SharpBicubic might prevent the over-smoothing effect of the filter.

best regards,
iago

[drizztcanrender]

iago said

BTW, are you gonna try convolution3d(0,8,8,8,8,3,0)?

Well i was thinking of choosing somewhat smaller values,bacause it is again a very clean source.I'm not aiming only on average quantizers but also on the picture being clear and not overly blurred.Maybe something like the ones vlad59 and Acaila suggested.

manono said

MPEG/MPEG quntizers-For 2 CDs, always (except anime). For 1 CD, it depends.

Well i think i would prefer he sharper effect in this test.Also this is a high paced movie so the quant type is going to be changing quite often,and that(if i have read correctly) change takes up space.

moanono said

Quants are 2-6/2-16-for testing, maybe. But ordinarily I limit the heck out of them. But different people do it differently.

Well then what do you say about 2-4/2-10??Just guessing here :)

manono said

SharpBicubic for resizing-Personally I don't touch it-I much prefer BicubicResize(xxx,xxx,0,0.5), but again, different people do it differently. I certainly wouldn't use it for a 1 CD rip, though.

Yeah i never use it either,too sharp for my eyes.I always go for Neutral Bicubic too.But since i'm going to put convolution3d before resizing i would like to use sharp so i can (maybe?) make the blurring less visible.

edit:
@iago

nice timing :)

[manono]

Hi-

iago-SharpBicubic might prevent the over-smoothing effect of the filter. -my mistake-I think you had written that earlier, but it had slipped my mind. I've done that myself when using spatial smoothers. However, I think an alternative would be to use the softer resize and to use weaker spatial settings in Convolution3d. For example, I'm testing (0,4,4,8,8,3,0) now to see how it looks.

drizz-what do you say about 2-4/2-10-there are plenty of threads about theory and methods, but I want the I-frame as high a quality as possible, and the P-frame settings will depend on the results of the Compress Test. But typically I'll use 2-3/2-5. If one of those Quant 10s sneaks in there, it'll screw up everything until the next keyframe is set. But that's just me-not making any recommendations. And don't tell Koepi:).

[drizztcanrender]

Ok i think i'll try using convolution3d(0,4,0,6,4,3,0).Prettty low maybe but i'm guessing it will give me a much clearer image.I still have about 2 hours before my other encode ends so i guess there is time for more suggestions.Also i'll try full matrix 1 and see how it goes.

[vlad59]

Always trying with Excalibur (because of the highly detailed pictures).
I only tested with Vdub by comparing frames, real test will come later.
So
Original (comp 30%)
convolution3D (0, 8, 8, 8, 8, 3, 0) is too blurry (comp 50%)

convolution3D (0, 8, 8, 8, 8, -1, 0) is a little less blurry but faster (comp 42%)

convolution3D (0, 4, 8, 4, 8, 3, 0) seems good lots of details are kept, faces look sometimes not natural (when the head use all the screen) (comp 42.2%) : I think if it's used before bicubicResize and with MPEG quantization : the result should be good.

convolution3D (0, 4, 4, 4, 4, 3, 0) : I can't see any visible change with my previous test ( ..., 4, 8, 4, 8,...) and the comp is 42% so I think there should be absolutly no noise in chroma.

That's my frame by frame impressions, tomorrow real XVID encodings.

[iago]

Originally posted by manono
... I want the I-frame as high a quality as possible, and the P-frame settings will depend on the results of the Compress Test. But typically I'll use 2-3/2-5. If one of those Quant 10s sneaks in there, it'll screw up everything until the next keyframe is set. But that's just me-not making any recommendations. And don't tell Koepi:). @manono

Maybe not the correct thread again, but it's still nice to discuss some filtering issues in relation to XviD (Don't tell Koepi!) ;). I use 2-6/2-16 for testing purposes usually, but concerning "real" encodes ;), imo an i-frame capping of 2-3 "might" also result in an overall quality loss due to higher average quantizer throughout the movie, because of consequtive i-frames, or just because of too many bits being spent on i-frames. Well, OK, you aim to prevent the quality loss by capping p-frames 2-5, but still there is the risk of having a "higher" average quantizer in the end (for example 3.5 instead of 3 let's say ;)), no?. (I generally use my old nandub routine ;), 2-4/2-8 for capping quantizers. But yes, as you say, that's just a "personal" choice or a "personal" way of doing things, and I respect it. I just wanted to share my opinions with you :).

with my best regards,
iago

[wing1]

@vlad59

Thanks for an excellent filter. I used this filter mostly on live source materials (not anime), and I have good result using this setting (1,4,5,3,4,2.8,0). If the source is noisy then I would bump all the numbers up by 1 or 2 except for the temporal influence setting which stays at 2.8. I would use -1 from all values if the source is exceptionally clean. The only problem with this filter for me so far is motion blur. Reducing the temporal influence setting to 2.5 seems to reduce that issue some; However, everything become quite sharp and compression becomes a problem (film grain-like result). 2.8 seems to be the middle ground between these two issues. As far as the filter's speed, it is great.

[vlad59]

@wing1

Huummm -1 for the tresholds, this case is not properly handled. It's either a hidden functionnality or a bug ....
I'll check that.

Anyway, you're almost the first to tweak "Temporal influence" and I'm happy you find it usefull. I got the same impressions as you : lowering the Temporal Influence keep some more details.

Thanks for your tests.

[bb]

@vlad59:
I think wing1 wanted to recommend to decrease his preferred threshold values by 1 for very clean sources, i.e. Convolution3D(1,3,4,2,3,2.8,0).

bb

[drizztcanrender]

Daaaaaamn!!
The tests are going to be a little delayed guys,my power supply died on me!!I mean i closed it and it never opened again!Anyway sorry for the delay,i'll be back as soon as i have them ready :devil:

[vlad59]

Originally posted by bb
@vlad59:
I think wing1 wanted to recommend to decrease his preferred threshold values by 1 for very clean sources, i.e. Convolution3D(1,3,4,2,3,2.8,0).

bb

Oooops, I should do like Koepi, never post while you're drunk ;)
Sorry for that.

@drizztcanrender

You shouldn't be so sad, it could be worse.
One year ago my power supply crashed while my PC was running and the results was :
Motherboard : dead
CPU (Duron 700) : dead
TNT2 : dead

My RAM and my HDD were safe hopefully.

I hope you'll get a new power supply fast, I'm waiting for your tests.

Regards

[drizztcanrender]

@vlad59

Hope that's not the case know.I'll either fix the safety gone bad or get me some money and by a 400W one.Or smth that can handle 10 days in a row :devil: (My old one clearly didn't :))

[iago]

Originally posted by bb
I think wing1 wanted to recommend to decrease his preferred threshold values by 1 for very clean sources, i.e. Convolution3D(1,3,4,2,3,2.8,0). I think I will give (1,3,4,2,3,2.8,0) and (1,4,5,3,4,2.8,0) a try (when I find the time) with the old, long, hard-to-compress Blues Brothers ;), aiming to fit it on 1 XCD.

regards,
iago

[drizztcanrender]

Back online!Finally my pc is working again :)
Now i can proceed with the tests!Ok since it is in my nature to do it, always, a little over the top i would like your opinion.I decided to do the following encodes from which, hopefully, i can get some insights about this filter.
First of all, i think this test-due to the once again very clean source-should not be about tweaking values of the filter,but about finding the general options of the codec that suit it best.So here are the tests i'm about to do(the clean encode is already on the way,so you have about 4 hours to tell me if i'm wrong about any of them ;)):


Movie: Crimson Rivers
Length: 1hour 41min 29sec
Desired video size: 795360kb
PAL Progressive 16:9

SmartRipper 2.40
DVD2AVI 1.76
Avisynth 2.05
VirtualDub 1.4.10

1)Clean encode:

Xvid - 2 Pass
Koepi's build 18082002
6 - Ultra High motion search precision
MPEG/MPEG Quantizers
no lumi masking
Neutral Bicubic Resize

2)Test encode 1:

Xvid - 2 Pass
Koepi's build 18082002
6 - Ultra High motion search precision
MPEG/MPEG Quantizers
no lumi masking
Sharp Bicubic Resize
Convolution3D(0,4,0,6,4,2.8,0) and Convolution3D(1,4,0,6,2.8,0) applied before resizing

3)Test encode 2:

Xvid - 2 Pass
Koepi's build 18082002
6 - Ultra High motion search precision
MPEG/MPEG Quantizers
no lumi masking
Sharp Bicubic Resize
Convolution3D(0,4,0,6,4,2.8,0) and Convolution3D(1,4,0,6,2.8,0) applied after resizing

4)Test encode 3:

Xvid - 2 Pass
Koepi's build 18082002
6 - Ultra High motion search precision
Modulated Quantizers
no lumi masking
Sharp Bicubic Resize
Convolution3D(0,4,0,6,4,2.8,0) and Convolution3D(1,4,0,6,2.8,0) applied before resizing

5)Test encode 4:

Xvid - 2 Pass
Koepi's build 18082002
6 - Ultra High motion search precision
Modulated Quantizers
no lumi masking
Sharp Bicubic Resize
Convolution3D(0,4,0,6,4,2.8,0) and Convolution3D(1,4,0,6,2.8,0) applied after resizing

6)Test encode 5:

Divx5.02
2 - Pass
GMC,B-Frame,Psychovisual -normal
Neutral Bicubic Resize


7)Test encode 6:

Divx5.02
2 - Pass
GMC,B-Frame,Psychovisual -normal
Sharp Bicubic Resize
Convolution3D(0,4,0,6,4,2.8,0) and Convolution3D(1,4,0,6,2.8,0) applied before resizing

8)Test encode 7:

Divx5.02
2 - Pass
GMC,B-Frame,Psychovisual -normal
Sharp Bicubic Resize
Convolution3D(0,4,0,6,4,2.8,0) and Convolution3D(1,4,0,6,2.8,0) applied after resizing

Quant distribution of the test encodes 5, 6 and 7 will be taken from the analyse.log files and will be outputed in the form of a graph(well maybe ;)).

So long and thanks for all the fish!!

[Emp3r0r]

eagarly awaiting results as i'm looking for a filter that does nothing but increase compressability no matter what the source.