"stair-step" jagged edges

[Frank K Abbott]

Is there anything I can do to fix stair-step type jagged edges? It's only on the edges. It's also a bit characteristic I think because the jaggies sort of "move around" the edges as people move or the camera pans.

[poisondeathray]

Do you mean aliasing?

Is this from progressive or interlaced footage? And how are you viewing it?

There are about a dozen antialiasing filters in this thread below. Some work better on particular sources than others, so you may have to experiment a bit
http://forum.doom9.org/showthread.php?t=153835

[nibus]

One of these usually works pretty well for me:


o = last
AssumeTFF().SeparateFields()
dbl = mt_Average( SelectEven().EEDI2(field=1,maxd=4),
\ SelectOdd() .EEDI2(field=0,maxd=4), U=3,V=3 )
dblD = mt_MakeDiff(o,dbl,U=3,V=3)
shrpD = mt_MakeDiff(dbl,dbl.RemoveGrain(11),U=3,V=3)
DD = shrpD.Repair(dblD,13)

dbl.mt_AddDiff(DD,U=3,V=3)

also:

AssumeTFF()
merge(SeparateFields().SelectEven().eedi2(field=1),SeparateFields().SelectOdd().EEDI2(field=0),0.5)


AssumeTFF()
SeparateFields()
merge(SelectEven().nnEDI(dh=true,field=1),SelectOdd().nnEDI(dh=true,field=0),0.5)

[Frank K Abbott]

Thanks nibus! I'm going to try those out.

@ poisondeathray

I think it's more of an alignment type of problem I'm having; something due to poor resizing I think. I tried nearly all the antialias functions but they didn't work at all for me. Even sangnom(aa=255) had no effect.

[Frank K Abbott]

@ nibus

Ok, so I tried all of those and nothing worked :(

Here's a little piece of the frame where it shows what I'm trying to fix. Around the frame edges.

http://img686.imageshack.us/img686/7439/jagged.png

[Guest]

Frank, answer the questions you are asked! What is the source material? How are you processing it?

Also post a small sample of the unprocessed source material that shows the problem.

[Frank K Abbott]

Source material is NTSC FILM (soft-pulldown) and I'm currently playing around in avsp to try to align/fix the edges. Final export will probably be x264.

[Guest]

You still don't answer fully. How are you processing it?

Where is the sample?

You can't get proper help if you withhold things that your helpers ask for.

[Frank K Abbott]

I'm using AviSynth to process it. http://www.sendspace.com/file/f1irmt

[Guest]

I'm using AviSynth to process it. LOL. I'll just turn on my mental telepathy to figure out what your script is.

Anyway, you can't fix that. The source is already ruined beyond repair. Looks like typical Bollywood garbage. Why waste your time?

[Frank K Abbott]

Right now my script is nothing other than the source line and crop. DGIndex already force filmed it.
But what I don't get is what are the jagged things on the edges called if it's not aliasing? And what is it due to? Thanks for looking at it all the same :)

[Guest]

It is aliasing. But it's so severe that it's really unfixable. It's due to a terrible encode job. Whoever made that is clueless.

Why are you so reluctant to tell us where this stream comes from?

[Frank K Abbott]

lol it's a really old and crappy dvd conversion from pal to ntsc that even I don't know where it came from. I was just cleaning up my house and I found it under a pile of rubbish. I was hoping I could fix it a bit. I'm pretty sure this was slowed down from 25p to 23.976 and then applied with a soft pulldown.

[poisondeathray]

I think most of the antialiasing filters detect "smaller steps" which is more typical. Your source has "long steps" so they don't even register as "aliasing" to the filter, resulting in little to no effect.

If you use santiag(3,3) you can derive marginal improvement (use the one gavino modified later in that same thread)

[Frank K Abbott]

santiag(3,3) (modded by Gavino) does make somewhat of a difference.

[Guest]

lol it's a really old and crappy dvd conversion from pal to ntsc that even I don't know where it came from. It's a TS file.

You are skirting on rule 6 territory here. I will await your explanation in PM and then decide whether to strike you. Thread closed pending clarification.