Custom Matrix comparison - ??? - V3

[kurt]

yes, it seems to be right now :-)

btw, i would like to see a custom matrices comparison about >2000 kbps (perhaps 2500 kbps?) ...
I encode always anamorphically in mkv on DVDs (2 movies on 1 DVD) and it would be great to compare the results at such bitrates...:D

[Soulhunter]

Originally posted by kurt

btw, i would like to see a custom matrices comparison about >2000 kbps (perhaps 2500 kbps?) ...
I encode always anamorphically in mkv on DVDs (2 movies on 1 DVD) and it would be great to compare the results at such bitrates...:D

I will, but only if you show me this five-point-palm-exploding-heart-technique... :D


@ Didée <- Yay, it works now !!!


Bye

[Soulhunter]

Question regarding the 900kbps encodes...

- Raw source or denoising ???

- If denoising, which kind ???


Question regarding all encodes...

- Cropping or a 1:1 encode ???

- Anamorphic encoding then ???


Tia n' Bye

[CeeJay.dk]

I think it's best to use settings that most people use to make 900kbps encodes.

I'd say use denoising .. I don't know what type is most popular , but I'd like to see undot used because AutoGK use that.

Most people crop the black borders and do not do anamorphic encodes.

[Audionut]

Removegrain (http://forum.doom9.org/showthread.php?s=&threadid=79751)

6000 frame clip (Clean PAL DVD) single pass, const. quant 2, with no filter's returned 2979kbps bitrate.
With Undot(), returned 2828kbps
With removegrain(), returned 2662kbps bitrate.
With removegrain() and unfilter(-5,-5) returned 2328kbps.

[Didée]

6000 frame clip (Clean PAL DVD) single pass, const. quant 2, with no filter's ...
With Undot ...
With removegrain() ...
With removegrain() and unfilter(-5,-5) ...

And with a simple

Deen(strong).MipSmooth(strong).RemoveGrain(strong).Dust(strong).unfilter(-150)

for sure you can get the bitrate low enough to make a nice 400 kbps comparison. Eventually with a little downsizing like Bicubic(50%, 1.0,0.0).

Seriously. It's out of question that for low-bitrate encodings the source should be filtered, sure. But with what filters, how much of them, and with what settings?

And, most importantly ... how much is the test going to say about the matrix used for compression? Well, not too much in fact. You'll mostly find out if a specific matrix reacts nicely to the results of a specific denoising strategy on a specific source. Not more, not less.

I'd go for a source that contains very little noise originally, and test the matrices without any filtering on that. (Movie "Vidoqc" springs to my mind.)

[Audionut]

Originally posted by Didée
And with a simple

Deen(strong).MipSmooth(strong).RemoveGrain(strong).Dust(strong).unfilter(-150)

for sure you can get the bitrate low enough to make a nice 400 kbps comparison. Eventually with a little downsizing like Bicubic(50%, 1.0,0.0).


WTF. Lay off the drugs. Or stick to making good custom matrices.

Almost everyone uses undot(). Removegrain() simply improves upon the ideas of undot().

Hands up who uses unfilter(-5,-5)

I once even seen a post by Koepi, recommending the use of unfilter. :devil:

Thanks a million Tom, really great work! Now I can do proper YV12 encoding again, your unfilter is needed e.g. for the second edition of 5th Element to get a compressability that makes some sense to put it on 1CD

To summerize, my idea of filtering and your idea of it, are 2 completly different things.

[Soulhunter]

Relax n' cool a bit down...

Ive not planed to use any filtering (transparency), but someone suggested to use denoising !!!

Just wanted to hear other peoples opinion about this... ;)


Bye

[Sharktooth]

The problem is the source has a lot of mosquito noise.
I did a test encode at 900kbits without filtering and the results are disappointing.
Even low bitrate matrices have problems.
The suggestion was to use some kind of temporal or wavelet denoisers to remove the background noise but leaving the details and film grain untouched.
Undot is not enaugh for that source. Vaguedenoiser seems to do the job pretty well but it is SLOW.
I will try unfilter and removegrain with the suggested settings and post the results.

EDIT:
Unfilter does not do the required job.
RemoveGrain(mode=x) is not enaugh even @ mode=9 and mode>4 drops too much details.
Convolution3D is pretty slow and does not do the required job well enaugh.
TemporalCLeaner(4,8) seems to do the job pretty well preserving grain and picture details.

any other ideas?

[SpikeSpiegel]

Originally posted by Sharktooth
The suggestion was to use some kind of temporal or wavelet denoisers to remove the background noise but leaving the details and film grain untouched.


I agree, because, in this case (->900kbps), the main goal of denoisers is to increase compressibility, so a temporal filter should be enough.

IMHO, TTempSmooth and RemoveDirt (if correctly used) are perfect for background noise; with default settings, TT won't erase details noticeably on motion areas, but RD is more aggressive.

[Didée]

Originally posted by Audionut
WTF. Lay off the drugs. Or stick to making good custom matrices.
...
To summerize, my idea of filtering and your idea of it, are 2 completly different things.

"Oh Lord, please don't let me be misunderstood."

[Sharktooth]

Originally posted by SpikeSpiegel
I agree, because, in this case (->900kbps), the main goal of denoisers is to increase compressibility, so a temporal filter should be enough.

IMHO, TTempSmooth and RemoveDirt (if correctly used) are perfect for background noise; with default settings, TT won't erase details noticeably on motion areas, but RD is more aggressive.

TTempSmooth does the job but even the faster version (TTempSmoothF) is too slow...
Undot + TemporalCleaner is still my preferred choice.
Im just not sure if Undot should be placed before or after TemporalCleaner.
Visually speaking i'd say "after" but theoretically it should not be so.

EDIT
My actual .avs looks like this:

# VIDEO SOURCE
Mpeg2Source("gladiator.d2v", idct=2 )


# CROPPING
# Crop(0,76,720,432)
Crop(6,76,704,432) # EDITED TO FIX PICTURE WIDTH


# FILTERS
# unfilter(-5,-5)
# removegrain(mode=9)
# Convolution3D(0,8,8,8,8,2.8,0)
# VagueDenoiser(threshold=0.8,method=1,nsteps=6,chroma= true)
# TTempSmoothF()
TemporalCleaner(5,10)
UnDot()


EDIT2
Rised TemporalCleaner settings to 5,10 after watching the source on a LCD monitor.

[SpikeSpiegel]

As you wish, anyway (I'm referring to everyone), as Didée's noticed (but maybe I misunderstood :D ), the purpose of this test shouldn't be "find the best hi-quality-filter-matrix-combo"; for example, using strong filters, we can make a HVS-best video look like onother encoded with H.263.
Ok, the example is a little excessive, anyway, I'm saying that in this case there's no reason to erase ALL noise (that can be used in the source-comp.file comparison, like details, to test the matrix aggressiveness), but only the noise that mostly reduces compressibility, the background one (and we already said how).

Another thought: to make the test more objective, we should make different tests, for example, a definition-test on a low-motion scene, then a noise-macro-test on a hi-motion one.

I hope this to be helpful (and not more confusing as always :devil: ), Soulhunter!

[Soulhunter]

Originally posted by SpikeSpiegel

I hope this to be helpful (and not more confusing as always :devil: ), Soulhunter!

Dont care... ;)


@Sharktooth

You PM-box is full !!!


Bye

[Audionut]

Please try this.

# VIDEO SOURCE
Mpeg2Source("gladiator.d2v", idct=2 )


# CROPPING
# Crop(0,76,720,432)
Crop(6,76,708,432)


# FILTERS
removegrain(mode=8,modeU=2)
fluxsmootht() t=temporal denoising only.
#Lanzcos4resize(xxx,xxx)
#After resizing I also add removegrain()
Unfilter(-5,-5)

Removegrain will only remove small grain.
Resizing will make larger grains smaller.
Hence using the very artifact free removegrain() after helps to remove the left over grain.

edit:
Could you please inform as to which chapter of Gladiator you are using for testing.

[Soulhunter]

Originally posted by Audionut

Could you please inform as to which chapter of Gladiator you are using for testing.

Chapter 19, same as in my last test... ;)


Bye

[Sharktooth]

Originally posted by Audionut
Please try this.

# VIDEO SOURCE
Mpeg2Source("gladiator.d2v", idct=2 )


# CROPPING
# Crop(0,76,720,432)
Crop(6,76,708,432)


# FILTERS
removegrain(mode=8,modeU=2)
fluxsmootht() t=temporal denoising only.
#Lanzcos4resize(xxx,xxx)
#After resizing I also add removegrain()
Unfilter(-5,-5)

Removegrain will only remove small grain.
Resizing will make larger grains smaller.
Hence using the very artifact free removegrain() after helps to remove the left over grain.

Too much filtering, too much slow :)
I dont want to remove grain... just noise. Film grain should remain.
Resizing is not an option.
However i will try fluxsmootht/removegrain without unfilter.

EDIT: It just does a very light temporal denoising. Removegrain is not doing the right job. The Undot()/TemporalCleaner(-5,-10) combo is producing better results... but i'm still playing with FluxSmoothT parameters.

EDIT2: TemporalCleaner/Undot combo provides better results.

EDIT3: The correct cropping is Crop(6,76,704,432). I should have been drunk when i typed 708... :D

[Sharktooth]

Originally posted by Soulhunter

@Sharktooth

You PM-box is full !!!


Bye
Wh00pz!
No longer:)

[JarrettH]

Almost done?:p ;)

[Sharktooth]

The actual choices are Undot + VagueDenoiser (slow) and Undot + TemporalCleaner (fast).
Both choices are good for the purpouse of removing mosquito noise but keeping details and grain.
Both choices produce high quality pictures.
Now the question is: to filter or not to filter?

EDIT: Undot + Vaguedenoiser were choosen for the test.

[Soulhunter]

Seems nobody cares... :(

So, I think we simply use filtering then !!!


Btw, Im already testing ya new matrix... :D

What stands the "HR" for ???


Bye

[Sharktooth]

Eh... damn hippies... when there's some work to do they all disappear (j/k) :D
HR stands for High bitRate.
V3UHR (Ultra High bitRate - good for Q2 encodings and bitrates over 2500) and V3LR (low bitRate) are coming too.

[Soulhunter]

@Sharktooth

You PM-box is full, again... ;)

[Sharktooth]

Doh! Ok... freed some space...

[olavitur]

Soulhunter: Have you by any chance already started making this comparison? I'm itching to compare some quantization matrixes...

[Soulhunter]

Yes, encodes are running... ;)


Bye

[Sharktooth]

I will start tonight :)

[Soulhunter]

Nice...

Btw, the 1600kbps encodes are done !!!

And this night I start the 2200kbps encodes... ;)


Bye

[Sharktooth]

Doh! I forgot AQ on ... restarting from scratch...

[olavitur]

Originally posted by Sharktooth
Doh! I forgot AQ on ... restarting from scratch...

Nice work Sharktooth... :)
Could you or Soulhunter post what setting you are using for this test?

[Sharktooth]

Xvid default settings except 2-passes and VHQ4 and Trellis ON.

[Soulhunter]

Originally posted by Soulhunter

I will, but only if you show me this five-point-palm-exploding-heart-technique... :D


No more need... (http://www.davidslife.com/journal/archives/000552.php) ;)


Bye

[kurt]

hehe, really nice link - couldn't explain it better :D

[((( atom )))]

err? - did i miss something here? i went roughly through the thread in search of new knowledege about what has happened on the matrix market in the last mont, the thread build up very promising and ends just like ...

??!?

help me out, please!

[Soulhunter]

Have a look here... (http://forum.doom9.org/showthread.php?s=&threadid=84375) ;)


Bye