new deblocker filter: SmoothD

[708145]

Hi there,

this is a preview release of my new filter. Several things are still not working but it should work alright without weird behaviour or crashes.

The filter is supposed to deblock frames while keeping high frequency detail. Version 0.1 is expected to be the first working version.

Please tell me if you find any bugs other than already mentioned in the attached html.

You can get it at
[edit]
project home
[/edit]

bis besser,
T0B1A5

[Mug Funky]

well.. it didn't crash VDM.

apart from that it doesn't really do anything (though that's explained in the docs).

good idea though. i tried to knock one of these up with DCTfilter and a bunch of yv12lut's, but obviously it wasn't an ideal solution

can't wait to see it with DCT alive and well.

[MfA]

You are aware that spp in mplayer/ffdshow does exactly what you want to do I assume?

[708145]

> You are aware that spp in mplayer/ffdshow does exactly what you want to do I assume?

Yep, but they keep very close to the paper. I'm gonna use adaptive_shifts and other shift patterns.

Also, using the same matrix as in the source helps for PSNR but is not the best choice for deblocking and other HVS considerations.

bis besser,
Tobias

[Leak]

Originally posted by MfA
You are aware that spp in mplayer/ffdshow does exactly what you want to do I assume?

Then again, using ffdshow as a pre-processor in AviSynth isn't exactly easy - I'd rather keep using MPEG2Source than use DirectShowSource if I can avoid it...

And yes, I like the results SPP deblocking in ffdshow produces, so I'd be happy to have an AviSynth filter that does something similar.


Oh, and don't try calling 708145 (your phone no.? ) - his phone ain't plugged in...

np: Bola - GLiNK (Soup)

[708145]

Then again, using ffdshow as a pre-processor in AviSynth isn't exactly easy - I'd rather keep using MPEG2Source than use DirectShowSource if I can avoid it...

And yes, I like the results SPP deblocking in ffdshow produces, so I'd be happy to have an AviSynth filter that does something similar.

The remaining problem will be to speed it up to realtime speeds. Should be possible but requires a lot of assembler skills.

Starting with V0.1 I will include the source (GPL) so others can contribute and speed-up. Many things are written very inefficiently.

Oh, and don't try calling 708145 (your phone no.? ) - his phone ain't plugged in...

Oh my god, you found out that I'm a professional lemming.
But it's not my phone number. See below for another hint.

bis besser,
T0B1A5

[Leak]

Originally posted by 708145
Oh my god, you found out that I'm a professional lemming.
But it's not my phone number. See below for another hint.

Okay, I see I wasn't thinking 1337 enough...

np: nix

[708145]

A new version is up!

It contains all code for V0.1 but does not compile
Maybe someone VC6 savvy can help me with this?

Futher steps are
1) speed optimizations (some are really easy and obvious)
2) test with all those different settings and
3) maybe even different matrices.

The final goal still is realtime! Maybe on sth like current 3GHz rigs. *hope*

bis besser,
T0B1A5

[krieger2005]

Hi,

i tried your filter in the previous version. I had a movie where big areas were flat (big areas with the same color and the same brightness). You can imagine that faces are very ugly when they have no contrast...

After using your filter i get on some these flat areas some points (little areas with different brightness). This is good because the flat areas are not sooo flat.

But i have now a question. Should i use the filter with an interlaced source:
- before deinterlacing
- before deinterlacing but with SeparateFields/Weave
- after deinterlacing

i tried to see the difference with the first and third point (before and after deinterlacing) and does not see any differences.

Well thanks for your filter

[708145]

Originally posted by krieger2005
Hi,

i tried your filter in the previous version. I had a movie where big areas were flat (big areas with the same color and the same brightness). You can imagine that faces are very ugly when they have no contrast...

After using your filter i get on some these flat areas some points (little areas with different brightness). This is good because the flat areas are not sooo flat.

Seems more like an error to me. But if you like this behaviour and it is not present in the final version anymore... well maybe I should offer a noDCT option (since V0.0.7 doesn't use DCT)

But i have now a question. Should i use the filter with an interlaced source:
- before deinterlacing
- before deinterlacing but with SeparateFields/Weave
- after deinterlacing

i tried to see the difference with the first and third point (before and after deinterlacing) and does not see any differences.

Well thanks for your filter

After deinterlacing is fine.
With separatefields it depends on what the filter sees: If it's twice the amount of frames but with half the vertical resolution then it should be OK.
But you shouldn't continue testing with V0.0.7! It really does not behave as intended, IOW too many things broken.

bis besser,
Tobias

[708145]

Well, V0.0.8 is up.

It
a) uses the Flat8 matrix instead of 6-of-9 and
b) doen't process U and V plane => still blocking!

enjoy und bis besser,
Tobias

[edit] typo fixed [/edit]

[Mug Funky]

hooray! it's nice.

i assume you've noticed, but num_shift wont go above 8, though adaptive shift will try to make it do so when quant is high enough.

also, if possible, DCT precision might need to be increased - there's some posterization happening (maybe changing the matrix would fix this too?)

[708145]

Originally posted by Mug Funky
hooray! it's nice.

i assume you've noticed, but num_shift wont go above 8, though adaptive shift will try to make it do so when quant is high enough.

The current formula for adaptiveshift is just an example. It will take some experience and testing with the filter and its behaviour to find out a good formula.

Anyways, I cannot spot a mistake about the range here:
num_shift=2+quant/5;
Thus even for quant 31 it should yield 8.2 which is 8 in (int).

also, if possible, DCT precision might need to be increased - there's some posterization happening (maybe changing the matrix would fix this too?)

I will exchange my own quant code with XviD's. Solving two problems in one turn:
a) bugs (XviD is well tested)
b) speed (it's faster than my code)

bis besser,
Tobias

[Dali Lama]

Hi,

I haven't had time to test this filter (or anything lately), but wouldn't it be excellent as a post-processor during playback? Especially since we would know what matrix and quant used during encoding?

-Dali

[708145]

Originally posted by Dali Lama
Hi,

I haven't had time to test this filter (or anything lately), but wouldn't it be excellent as a post-processor during playback? Especially since we would know what matrix and quant used during encoding?

-Dali

Sure Actually I'd love to use it as a pp. But even spp - which is much more optimized - is too slow for most resolutions/computers.

The idea behind the fact that I dubbed (or will dub, actually) the semi-final feature-complete version V0.1 is that it is about a factor of 10 apart from a V1.0 performance wise!

V1.0 is supposed to process the video in real-time. Everyone is invited to help optimizing of course.

bis besser,
Tobias

[AS]

While spp might be too intensive for real time pp purpose, any chance of getting it as an avisynth filter, or as an addition to core smoothD function?

[Leak]

Originally posted by AS
While spp might be too intensive for real time pp purpose, any chance of getting it as an avisynth filter, or as an addition to core smoothD function?

Well, when SmoothD is feature complete it's meant to do exactly what SPP does...

np: Triola - Distel (Im Fünftonraum)

[708145]

A new preview version is up: V0.0.9pre

Source only again since I need help with compiling.
Please PM me for ideas/solutions. Thank you.

bis besser,
Tobias

[morsa]

Are there any image samples available?

[708145]

Originally posted by morsa
Are there any image samples available?

Well, take the following examples with a grain of salt since they were produced with the "reference software" provided by A. Nosratinia.

original
processed

bis besser,
Tobias

P.S.: Thanks for the input on my compiler problems. If I'm lucky it'll work today.