A Comparison between Cnr2().TemporalSoften(1,5,0) and convolution3d(1,10,22,8,8,0)

[iago]

Hello everybody,

In terms of better organization, I find it more suitable to start a new thread for this specific test, which was formerly discussed in http://forum.doom9.org/showthread.php?s=&postid=169149#post169149.

TEST SETUP

Movie: The Blues Brothers (1980) - NTSC 1
2hr-27min/512*272/SimpleResize
Aimed Video Size: 650000kb (635mb)

SmartRipper 2.40
DVD2AVI 1.76 (ForcedFILM 29.97 -> 23.976)
Gordian Knot 0.26 beta (crop/resize)
VirtualDub 1.4.10 (fast recompress)

XviD 2pass
Koepi's 18082002-1 build
1stPass: 6.UltraHigh/H.263/Lumi enabled
2ndPass: 6.UltraHigh/MPEG/Lumi enabled/Min-Max I-frame:2-6/Min-Max P-frame:2-16/default Alt.CC/all the rest default
Maximum-Minimum I-frame interval: 250-1 (both passes)

Avisynth scripts:
-----------------

(1) (Avisynth 2.04+)

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\cnr2.dll")
MPEG2source("D:\BLUSBRO\BLUSBRO.d2v").crop(3,7,714,460).SimpleResize(512,272)
Cnr2()
TemporalSoften(1,5,0)
Trim(0,205210)+Trim(205211,0).TemporalSmoother(10)

(2) (Avisynth 1.06b)

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\convolution3d.dll")
MPEG2source("D:\BLUSBRO\BLUSBRO.d2v").crop(3,7,714,460).SimpleResize(512,272)
Convolution3d(1,10,22,8,8,0)
Trim(0,205210)+Trim(205211,0).TemporalSmoother(10)

---------------------------------------------------------------------

1st Pass Results:
------------------
Speed:
(1) is ~25-30% faster than (2)

Compressibility - 1stPass/2ndPass Ratio:
(1) Compressibility: 39.1% (in GKnot) - 1stPass/2ndPass Ratio: 1624mb/635mb: 2.55
(2) Compressibility: 41.7% (in GKnot) - 1stPass/2ndPass Ratio: 1522mb/635mb: 2.39

---------------------------------------------------------------------

The second pass results will be posted (together with the screenshots) as soon as finished.

best regards to all,
iago

[ronnylov]

Why don't you test both scripts in Avisynth 2.04? It's easier to campare the speed if the same version of avisynth is used.

I have another filtering idea.

cnr2().Convolution3d(1,10,0,8,0,0)

If I'm correct the script above will filter chroma only with cnr2() and luma only with convolution3d. But I'm not sure that convolution3d is faster when the chroma tresholds are set to 0.

How much will the compressibility improve if both methods are used at the same time, like this? And what about the quality?

cnr2().temporalsoften(1,5,0).Convolution3d(1,10,22,8,8,0)

or

cnr2().temporalsoften(1,5,0).Convolution3d(1,10,0,8,0,0)


Just some ideas. I have not done any tests yet.

[vlad59]

@ronnylov

No, Convolution3D won't be faster if only luma is filtered. The first version were faster in this case but since beta (MMX & iSSE optimizations) you gain nothing when filtering only luma :( :( .

[iago]

@ronnylov

Originally posted by ronnylov
Why don't you test both scripts in Avisynth 2.04? It's easier to campare the speed if the same version of avisynth is used. Well, as it was pointed out before, speed is not the "main" concern in the test. And the reason why I switch(ed) between different versions of Avisynth is that I'm too tired of getting "access violation" messages, which causes a lot of delay in the test. (However, interestingly, I got the error message both with 2.04+ (script 1) and 1.06b (script 2) in the second passes.)

iago

[Marc FD]

dunno what mean the Conv3D params, but they might be more aggressive than Cnr2().TemporalSoften(1,5,0) ...
and if speed is really not you concern you should try Conv3D agains pseudo 3D filtering. i think of MAM here.
MAM(Cnr2().temporalSoften(1,10,0),SmootherHIQ(??)) could give good results.
you could use some aggressive settings with MAM, because ghosting from Temporal filtering would be avoided and blurring of Spatial filtering would be only on Moving things... (less noticable)

just my thoughts to do a real comparison between 3D filtering methods

[vlad59]

@MarcFD

It's hard to compare the Convolution3D paramaters and Cnr2().TemporalSoften(1,5,0) because the algo are different and the parameters have not the same meaning.

I personnaly have tested Cnr2 once (~ a month ago) and I got some chroma ghosting and I have not made any test till that (no time, too much work). TemporalSoften(2) is the same : I never made any test with it.

I only regularly use TemoralSmoother (2,1) and now convolution3D ;) .

I should make some tests with other filter (especially MAM), this weekend if I have enought time :D

[iago]

@Marc FD

Yes, of course other tests can also be done later, but for the time being I really want to "see" the visual and have a look at the debugview analyzer results of the second pass with these two scripts in this hard-to-compress case, which I doubt that I will be able to finish anyway :). The second pass of script(2)/avisynth1.06b has already crashed two times giving the attached (access violation) error message :).

Now I'm going with the second pass of script(1)/avisynth2.04+. I hope it ends with no crash, and I try the second pass of script(2)/avisynth1.06b again.

so long,
iago

[vlad59]

That's stange iago, Koepi is using Convolution3d to encode matrix and he didn't report any avisynth bug.

And I personnaly never had this problem when testing Convolution3D.

I'll check that with him.

[iago]

@vlad59

I read Koepi's reply in the related thread. He says he's using Avisynth 2.02. My second pass of script(1)/avisynth2.04+ is about to finish, so far happily with no problems.

Possibly, to avoid a similar crash with 1.06b and waste of time again, I'll start the second pass of script(2)/convolution3d(1,10,22,8,8,0) with Avisynth 2.04+. And if it doesn't work I'm going to try 2.02. (Hence, I guess I'll have verified the version of avisynth that works without causing any problems with convolution3d on "my" system ;).)

best regards,
iago

PS: an important mistake corrected here: paragraph2 -> script(2)/convolution3d(1,10,22,8,8,0)

[iago]

Hello again,

First of all, my second pass with script(2)/convolution3d(1,10,22,8,8,0) using Avisynth 2.04+ crashed too, giving the same (access violation) error message.

However, until the crash, the second pass of script(2) created a 230000kb file that will be sufficient imho to compare the jobs of the two filters.

Yet, for the same reason (at least for the time being) it will not be possible to compare the debugview analyzer results, but it is possible to say (looking at the first pass results and script(1) second pass results) that average quantizers and compressibility wouldn't differ a lot, both encodes generating an average quant. of ~4.5 with a compressibility value of ~40%.

I'm preparing and will send the screenshots very soon for visual comparison between the results that the two scripts produced.

regards,
iago

[iago]

scripts:

(1)
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\cnr2.dll")
MPEG2source("D:\BLUSBRO\BLUSBRO.d2v").crop(3,7,714,460).SimpleResize(512,272)
Cnr2()
TemporalSoften(1,5,0)
Trim(0,205210)+Trim(205211,0).TemporalSmoother(10)

(2)
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\convolution3d.dll")
MPEG2source("D:\BLUSBRO\BLUSBRO.d2v").crop(3,7,714,460).SimpleResize(512,272)
Convolution3d(1,10,22,8,8,0)
Trim(0,205210)+Trim(205211,0).TemporalSmoother(10)


results:
Screenshots attached below (hope it will be approved soon).
And just a few comments:
1) In really hard-to-compress cases (like this one) and aiming for 1CD, even filtering cannot produce an acceptable/watchable quality preserving enough details and not producing artifacts. You increase compressibility by filtering: and the details are gone as well as artifacts may be introduced. You don't use filtering: the result is very poor with a high average quantizer and many blocks throughout the movie. As a result, imho, sometimes the only solution is "shooting for one more CD". (I remember discussing the same issue with Marc FD a long while ago in one of the XviD threads ;))
2) In the screenshots, (though many comments can be made), especially one thing is very visible: the artifacts introduced by script(1) at keyframes.

And yes, to cut it short, here are the screenshots and waiting for your comments.

best wishes,
iago

[ACClarke]

(1)
good details but colors problems

(2)
true colors but details problems

so.........

[soulfx]

The reason for the color problems and artifacts in script 1 is do to the use of temporal filters (temporalsoften for sure... I think cnr2 is time based too, but not sure... I know cnr2 will produce a slight ghosting affect though). TemporalSoften is an awesome filter to retain detail, but really stinks during motion and scene changes.

Script2 will do a better job during motion scenes, but will really dump detail during still motion.

To get the best of both worlds use Marc's awesome MAM filter. Use the filtering scheme used in script1 as your temporal and the filtering scheme in script2 as your spatial. This will cause temporalsoften to be used during low motion (great detail) and during motion (including scene changes) Convolution3d will be used.

Worth a shot anyways :)

Peace,
SoulFX

[iago]

@soulfx

I haven't used MAM before (actually I don't use much filtering anyway and prefer to go for one more CD in such cases ;) but (LOL!) this time I'm determined to try my best with filtering :)) when I read the readme file attached, I think such a script should be used:

------------------------------------------------------
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\mam.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\cnr2.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\convolution3d.dll")
MPEG2source("D:\BLUSBRO\BLUSBRO.d2v").crop(3,7,714,460).SimpleResize(512,272)
MAM(Cnr2().TemporalSoften(1,5,0),Convolution3d(0,10,22,8,8,0))
Trim(0,205210)+Trim(205211,0).TemporalSmoother(10)
------------------------------------------------------

Right?

thanks,
iago

[theReal]

iago,

I got that access violation message recently with avisynth 2.03 (which I'm using because it deals perfectly with doubled frames in mjpeg captures, which 2.04 does not).

The source had been captured in HuffYUV and first I was also using convolution3d - but then I disabled all filters in avisynth (only opened the avi with avisource()) and still got the access violation, always at the same few frames (frames 52-60 in that special file).

I couldn't figure out what the problem was with these frames, they showed perfectly normal in all other applications. I guess it's more an avisynth problem than a convolution3d problem - do you only get it with this filter?

[vlad59]

Hi

Just by looking at the first test frame (11242), it's obvious that Convolution3d (x, 10, 22, x, x , x) is way too strong for normal film.

I only tested this strong setting with anime and never tested with normal film.

So with film you should keep the Convolution3d (0, 8, 16, 8, 8 , 0) or maybe Convolution3d (1, 8, 16, 8, 8 , 0) or even 8, 8 if you really want to keep details.

Anyway thanks for the test iago !

[iago]

Originally posted by vlad59
...it's obvious that Convolution3d (x, 10, 22, x, x , x) is way too strong for normal film. I only tested this strong setting with anime and never tested with normal film. So with film you should keep the Convolution3d (0, 8, 16, 8, 8 , 0) or maybe Convolution3d (1, 8, 16, 8, 8 , 0) or even 8, 8 if you really want to keep details. Anyway thanks for the test iago ! It's OK vlad59, I'm glad at least this test helped to reach this important conclusion, which imho is an important step :). (Maybe you consider updating the readme file for the next version of this promising filter ;).

Originally posted by theReal
...guess it's more an avisynth problem than a convolution3d problem - do you only get it with this filter?! Currently I'm using avisynth 2.04+ (15082002), and I got the error message not only with convolution3d but once with Cnr2().TemporalSoften(1,5,0) too. I agree with you and I don't think it's related to convolution3d either. I'll try the new avisytnh 2.04+ (21082002) from CVS soon.

best wishes,
iago

[Marc FD]

@ thereak and iago

you're really not lucky
dunno what's you config, but i never had acces violations in avisynth with other than mine filters. and when it's with one of my filter, it's always my fault :rolleyes:

(but i didn't made very long encodes (>4 hours) a long time ago...)

using avisynth 2.0.4 build from SF (sourceforge)

[iago]

And (for those who are interested) even more screenshots of the same frames to compare with the previous ones, this time with the following script:

LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\mam.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\cnr2.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\convolution3d.dll")
MPEG2source("D:\BLUSBRO\BLUSBRO.d2v").crop(3,7,714,460).SimpleResize(512,272)
MAM(Cnr2().TemporalSoften(1,5,0),Convolution3d(0,10,22,8,8,0))
Trim(0,205210)+Trim(205211,0).TemporalSmoother(10)

regards :),
iago

[Sigmatador]

and what about Compressibility and 1stPass/2ndPass Ratio with MAM ?

[iago]

@Sigmatador

I didn't look in GKnot but 1stpass/2ndpass ratio was within the same range as the previous ones, a bit higher than 2.4 as I remember, 'cause I deleted the stats file after I saw the still-not-satisfactory visual results of the second pass :).

regards,
iago

[Sigmatador]

first test is finished. encoding with cnr2 script and the result is good. good compressibility and detailled video. but some bug's color, maybe ghosting effect ... but only on fast movement, low movement are nearly perfect ^^

i'm going to start the test with convolution script but can someone tell me where can i download the last version of convolution3d.dll plz ?

[iago]

@Sigmatador

It is hosted here: http://hellninjacommando.com/con3d/

iago

[Archiwa]

And this script do good to me .


#################################################
# ASYNTH v2.04 De-Interlace + Noise reduction #
#################################################

LoadPlugin("C:\Programs\avisynth\Convolution3D.dll")
LoadPlugin("C:\Programs\avisynth\TomsMoComp.dll")
LoadPlugin("C:\Programs\avisynth\mam.dll")
LoadPlugin("C:\Programs\avisynth\MPEG2DEC.dll")
LoadPlugin("C:\Programs\avisynth\SmoothHiQ.dll")
LoadPlugin("C:\Programs\avisynth\Cnr2.dll")

AVISource("D:\...")

Convolution3D(8,16,0)

TomsMoComp(1,0,1)

MAM(TemporalSmoother(3,1),SmoothHiQ(3,10,0,75,1))

Cnr2()

SmoothHiQ(3,0,15,75,1)

Tweak(hue=9.0)


can try ?


thanks.

[Suzahara]

1st Pass Results:
------------------
Speed:
(1) is ~25-30% faster than (2)

There is a slight memory problem in 2.04, it is a lot faster if you add a SetMemoryMax(40), this has been discussed in other threads and makes 2.04 a lot faster, 2x as fast for me in fact.