Mpeg artifact around edges

[karasu]

I'm not very experienced with avisynth filters, maybe you can help me.

This source have a lot of mpeg blocks around the edges :
http://gloomydream.net/bazar/dvd.png

I have good results using
fft3dfilter(bw=16, bh=16, ow=8, oh=8, bt=3, sigma=3, plane=4, sharpen=1, scutoff=0.5, ncpu=2)

but the background textures and the smoke effects becomes blurry with sigma 3.
Is there a better way to remove these artefacts?

[Sagekilla]

sigma=3 is a bit of a high value to be using for FFT3Dfilter -- you can try setting sigma=2, sigma2=1.5, sigma3=1.5, sigma4=1.5 so it doesn't filter the lower frequencies as intensively and only focuses on the high frequencies, I believe. I'm not sure what the cutoffs are but generally sigma = high frequencies (like those tiny little artifacts) and sigma4 = lowest frequencies (blob sized artifacts)

[karasu]

Perhaps FFT3D is not the best filter here.
Here's my results with different settings :

source (http://gloomydream.net/bazar/dvd.png)

fft3dfilter(sigma=2, sigma2=1.5, sigma3=1.5, sigma4=1.5, ncpu=2) (http://gloomydream.net/bazar/fft3d5.png)

fft3dfilter(sigma=3, ncpu=2) (http://gloomydream.net/bazar/fft3d7.png)

And a surfaceBlur form Photoshop (http://gloomydream.net/bazar/surfaceblur.png), the background is destroyed, but it works pretty well in flat area.

Any idea to get better results?

[Adub]

Take a look at Msmooth. It works wonders for anime. Also dfttest instead of fft3dfilter. It's slower, but better.

Last but not least, if you have years to encode, TNLmeans.

[pitch.fr]

I guess there's no fast way to deblock this sort of things on regular movies ?
so I could use it in realtime in ffdshow :D

[Adub]

Well, for one, I don't see any blocking in this source. But if you do have sources with blocking in them, then Deblock_QED works pretty well, and I think it runs faster than real time.

But for these artifacts, Msmooth works pretty well, and it's not overly slow. You might be able to get it running in realtime or faster on that quad of yours. I am not sure, as I usually use a bunch of other filters as well, so my experiences are skewed.

[Dark Shikari]

For realtime cleanup, SPP deblocking in FFDshow with "Soft Threshold" does the job; its basically like dfttest except not as good but an order of magnitude faster. Adjust the top left slider (lower is faster) until its realtime.

[Adub]

What exactly is SPP deblocking? SuperPostProcessor or something?

[Dark Shikari]

What exactly is SPP deblocking? SuperPostProcessor or something?Its a deblocker that uses overlapped DCTs. Regular mode drops all coefficients below $threshold to 0, and Soft Threshold drops all coefficients by a certain amount except the DC coefficient.

[pitch.fr]

I just tried Deblock_QED(25), it said "you cannot use crop to enlarge or shift a clip" ?!

yeah, I was talking about this ugly MPEG1 blocking around edges....

you mean "SPP deblocking" in the "post processing" filter of ffdshow ?
simply check "SPP~" and "soft threshold" and then play with the "processing strengh" slider ?

this looks blurry :D

[Didée]

... SPP deblocking in FFDshow with "Soft Threshold" ... Adjust the top left slider (lower is faster) until its realtime.
For me, that's the slider position to the very very left. :D

[Mystery Keeper]

If you don't mind color conversion - VirtualDub's MSU_Cartoon_Restore.

[Dark Shikari]

I just tried Deblock_QED(25), it said "you cannot use crop to enlarge or shift a clip" ?!

yeah, I was talking about this ugly MPEG1 blocking around edges....

you mean "SPP deblocking" in the "post processing" filter of ffdshow ?
simply check "SPP~" and "soft threshold" and then play with the "processing strengh" slider ?

this looks blurry :DAdjust the strength downwards; I find that I have to drop it a lot for it to be to my satisfaction.

Soft threshold off will improve sharpness but it won't be nearly as powerful.

[karasu]

Where can I found Msmooth?
I can't get dfttest working. I get a scrambled red message, Did I miss something?

TNLmeans is great (http://www.gloomydream.net/bazar/tnlmeans.png), but very slow ;-)

[jeffy]

@karasu: Have you ever heard of Google?
http://www.google.com/search?q=msmooth

[karasu]

Google? what is that? ... OMG it's so great!! ;-)

thanks for msmooth.

But msmooth is too strong for background details, event with low threshold and strength=1.

With SPP I get good results, TNLMeans is slightly better but way slower.

[karasu]

I upgraded to FFDshow rev2033 (Jun 27 2008) and now I can't get SPP deblocking working. Is this a bug?

[pitch.fr]

I also don't understand how to use it, but nvm.

I guess the only way to remove these artifacts is to blur things up.....there's no such things as miracles.

[karasu]

Theses artefacts only appear near edges, I don't know how to do that but here is a possible way:

- detect the edges => a
- Blur and level a to "grow" the edges => b
- b-a => c
- apply a surface blur with c as mask


(I hope this have sense for someone)

[karasu]

The result of my "algo" with photoshop :

http://www.gloomydream.net/bazar/photoshop2.png
need fine tuning but artefacts and white outline have gone.

[Adub]

Nah, you have to turn the threshold up on Msmooth to "turn it down". Use the mask parameter to see what you are affecting and then play with the threshold, you will see what I am talking about.

[karasu]

I used the mask to see whats happens, but the background details are more affected than the artifacts...

[LaTo]

Spline36resize(2*720,2*576)
Sangnom().TurnLeft().Sangnom().TurnRight()
Spline36resize(720,576)
Remove most of artefacts...

[Mystery Keeper]

Spline36resize(2*720,2*576)
Sangnom().TurnLeft().Sangnom().TurnRight()
Spline36resize(720,576)
Remove most of artefacts...

That's antialiasing. It won't remove Gibbs.

[Adub]

The mask parameter doesn't show you the artifacts. It shows you the edges that are detected from the current setting of threshold. The lower the threshold, the more amount of edges detected. set it somewhere between 8-12 and see how it looks. Msmooth works great for these type of artifacts, as I use it all of the time.

If you provide a sample, we can show you what is possible to be achieved.

[LaTo]

That's antialiasing. It won't remove Gibbs.

Yes, but it help to remove mpeg artefacts...

Original
http://gloomydream.net/bazar/dvd.png

AA
http://img135.imageshack.us/img135/394/dvd2gs1.png

But halos are always here, maybe DeHalo_alpha can fix this...

[Mystery Keeper]

http://ipicture.ru/uploads/080829/8073/thumbs/jz238pV5wt.png (http://ipicture.ru/Gallery/Viewfull/6292933.html)

LoadVirtualDubPlugin("E:\Program Files\VirtualDub\plugins\msu_cartoon_restore.vdf","MSUCartoonRestore", 0)
#ConvertToRGB32.MSUCartoonRestore("simple", Smoothness, Threshold).COnvertToYV12
ConvertToRGB32.MSUCartoonRestore("simple", 1, 70).COnvertToYV12
SharpAAMC()

[Blue_MiSfit]

I demand more fft3dfilter/gpu :D

It should clean up those flat areas nicely!

Maybe some de-haloing too?

~MiSfit

[Adub]

Again, this is hard to do/tell on a single frame, which is why I asked for a small sample.

[Blue_MiSfit]

I like the MSU Cartoon restoration results. Either the filter itself or the color conversion required seems to reduce saturation a bit, but it does a nice job.

[Adub]

Hmm... I wonder how it compares against Msmooth, especially in terms of detail retention. I never use any of the MSU stuff because I don't trust them (don't ask me why, I just prefer our home grown avisynth filters right here on the forum).

[karasu]

Here is a vob sample (http://gloomydream.net/bazar/sample.vob)

Merlin7777: I'm interested in your msmooth results, I can't get something decent with it.

Mystery Keeper : For me the background is too blurred.

LaTo : You have taken the photoshop filtered picture as source.


Here's the artefacts I want to get rid of :
http://gloomydream.net/bazar/artifacts.png

[LaTo]

LaTo : You have taken the photoshop filtered picture as source.
No, just an error when I made the post... FIXED

[karasu]

Ok LaTo, your filter is the most efficient in my case.

[Jeremy Duncan]

karasu,

can you please post a unprocessed bmp picture showing the artifact you want to get rid of?
not resized or manipulated by any avisynth filters. :)

also, what size do you want it to be when it's done?

do you want fast encoding or slow? :)

[martino]

He posted a vob sample. That's much better than just a bmp/png w/e else...

[Jeremy Duncan]

vob sample link is broken.
don't think i tried that? :)

[karasu]

vob sample link is broken.
don't think i tried that?

The link isn't broken. And if you read my first post you will find a uncompressed PNG of my source. I don't want any resizing.

[Jeremy Duncan]

yes, the link is broken.
try rapidshare.

[talen9]

I just tried the link and it works ... seems like the problem is at yours side, J.

[Jeremy Duncan]

if i click the dl vob link it says dl'ing, but then a quicktime window pops up once the dl is finished and there's no option to save the vob to desktop.
smells fishy to me. i don't think there ever was a dl link that worked differently than it does now. maybe pirated?
i'm not even sure the original picture was unprocessed.

[talen9]

I completed the download (I'm using Google Chrome as the browser) and nothing of what you said happened ... maybe your browser is (wrongly) configured to open .VOB files with QuickTime and the window closed simply because it didn't support it? :)


BTW: i recognized the movie (Akira); I own the original PAL DVD, and here's the same frame directly from VTS_01_2.VOB (screenshot grabbed via VDubMod):

http://img187.imageshack.us/img187/7514/akira1wk1.png

[45tripp]

Jeremy,

right click, 'save raget/link as'

[ChrisW77]

I tend to like over-smoothing my anime, just for personal tastes, although more on older anime from the 80's and early 90's.

Probably a bit too smoothed over, but I always use a lot of fft3dgpu. Could be sharpened with a bit of LSF, perhaps.

http://i33.tinypic.com/rcvlo2.png

[Jeremy Duncan]

DeHalo_alpha()
dfttest(tbsize=1)



function DeHalo_alpha(clip clp, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()
uv = 1
uv2 = (uv==3) ? 3 : 2

halos = clp.bicubicresize(m4(ox/rx),m4(oy/ry)).bicubicresize(ox,oy,1,0)
are = yv12lutxy(clp.expand(U=uv,V=uv),clp.inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
ugly = yv12lutxy(halos.expand(U=uv,V=uv),halos.inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
so = yv12lutxy( ugly, are, "y x - y 0.001 + / 255 * "+LOS+" - y 256 + 512 / "+HIS+" + *" )
lets = maskedmerge(halos,clp,so,U=uv,V=uv)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.lanczosresize(m4(ox*ss),m4(oy*ss))
\ .logic(lets.expand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"min",U=uv2,V=uv2)
\ .logic(lets.inpand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"max",U=uv2,V=uv2)
\ .lanczosresize(ox,oy)
them = yv12lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

return( them )
}

function m4(float x) {return(x<16?16:int(round(x/4.0)*4))}

i'm sorry, you were all right. when i right clicked i could save to me desktop.
again i'm very sorry. :)

[karasu]

BTW: i recognized the movie (Akira); I own the original PAL DVD, and here's the same frame directly from VTS_01_2.VOB (screenshot grabbed via VDubMod):



Interesting, I own the French "special edition" DVD. Looks like a bad re-encoding. On your frame, we can see more on the left, on my DVD it's covered with black band. Your source doesn't seem to have mpeg artifact like mine.

(can I ask you in private if you can send your vobs on my FTP?) :)

ChrisW77 : A bit too smooth for me, but interesting result, what's your settings?

[talen9]

Hmm I don't think that's something we should talk about, don't you think? ;)

Anyway, seems like the master used for your DVD has been sharpened (at least, comparing it with mine) and it's likely that the artifacts you're seeing descend more from this (ringing/haloing) than from MPEG compression ... Jeremy Duncan's suggestion should bring you a good result, maybe with a slight tinkering of the parameters if the default doesn't satisfy :)

[karasu]

Jeremy : Thanks for the de-halo function, The first who works for me.

I get the dfttest plugin from here: http://bengal.missouri.edu/~kes25c/
But I only get a scrambled red message when I try to play the avs. How to get it working?

Edit: I put libfftw3f-3.dll in my system32 folder and now it's OK.

[ChrisW77]

The thing is, I can never remember Akira looking so bad, and certainly so sharp, thats why mine's a little smoother, and yours looks like it's been sharpened too much.
My sources of Akira are VHS and R2 DVD, and they don't look so sharp as that.
Perhaps, instead of trying to fix this, you could get the Remastered DVD ?
Here in the UK, it's around £11 for the 2-CD set. I am curious as to what it looks like, and whether they did a good job of remastering.

http://www.play.com/DVD/DVD/4-/119925/Akira-The-Ultimate-Collection/Product.html

[martino]

I doubt the PAL version would have less issues than the NTSC one...

[karasu]

ChrisW77 : I like sharpness, but here, it's obviously oversharpened.
Can you post a picture of the same frame from your DVD?

Here in switzerland (and in France) Akira is out of stock for years now.

[karasu]

Here's my final result:
http://www.gloomydream.net/bazar/final.png

the script:

loadplugin("C:\Program Files\AviSynth 2.5\plugins\FFT3DFilter\fft3dfilter.dll")


Spline36resize(2*704,2*576)
Sangnom().TurnLeft().Sangnom().TurnRight()
Lanczos4Resize(704,576)



DeHalo_alpha()
unfilter(-4,-4)
FFT3DFilter(bt=-1, sharpen=0.8)



function DeHalo_alpha(clip clp, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()
uv = 1
uv2 = (uv==3) ? 3 : 2

halos = clp.bicubicresize(m4(ox/rx),m4(oy/ry)).bicubicresize(ox,oy,1,0)
are = yv12lutxy(clp.expand(U=uv,V=uv),clp.inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
ugly = yv12lutxy(halos.expand(U=uv,V=uv),halos.inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
so = yv12lutxy( ugly, are, "y x - y 0.001 + / 255 * "+LOS+" - y 256 + 512 / "+HIS+" + *" )
lets = maskedmerge(halos,clp,so,U=uv,V=uv)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.lanczosresize(m4(ox*ss),m4(oy*ss))
\ .logic(lets.expand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"min",U=uv2,V=uv2)
\ .logic(lets.inpand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"max",U=uv2,V=uv2)
\ .lanczosresize(ox,oy)
them = yv12lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

return( them )
}

function m4(float x) {return(x<16?16:int(round(x/4.0)*4))}

I dont know If Lanczos4Resize is sharper than Spline36resize, my eyes can't tell.

Thank you all for helping me! :)

[*.mp4 guy]

http://img61.imageshack.us/img61/4831/slowig9.png (http://imageshack.us)
http://img61.imageshack.us/img61/slowig9.png/1/w720.png (http://g.imageshack.us/img61/slowig9.png/1/)

[ChrisW77]

Ok, last go, slightly sharper, less smoothing.

http://i38.tinypic.com/105azgk.png

[karasu]

ChrisW77: Great! :script:

(however, The frame I used is the next one, look at the fingers of the girl)

[Comatose]

dehalo_alpha using masktools 2
function DeHalo_alpha(clip clp, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()
uv = 1
uv2 = (uv==3) ? 3 : 2

halos = clp.bicubicresize(m4(ox/rx),m4(oy/ry)).bicubicresize(ox,oy,1,0)
are = mt_lutxy(clp.mt_expand(U=uv,V=uv),clp.mt_inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
ugly = mt_lutxy(halos.mt_expand(U=uv,V=uv),halos.mt_inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
so = mt_lutxy( ugly, are, "y x - y 0.001 + / 255 * "+LOS+" - y 256 + 512 / "+HIS+" + *" )
lets = mt_merge(halos,clp,so,U=uv,V=uv)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.lanczosresize(m4(ox*ss),m4(oy*ss))
\ .mt_logic(lets.mt_expand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"min",U=uv2,V=uv2)
\ .mt_logic(lets.mt_inpand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"max",U=uv2,V=uv2)
\ .lanczosresize(ox,oy)
them = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

return( them )
}

function m4(float x) {return(x<16?16:int(round(x/4.0)*4))}