nnedi3 upsize artifacts

[feisty2]

well, it seems nnedi3 doesn't work so well on areas full of small and dense details like fallen leaves on the ground
here's the original frame it's 720x480 sd content from a dvd I bought, I'd like to upscale it for hdtv display
original
http://thumbnails104.imagebam.com/30606/2596c6306059524.jpg (http://www.imagebam.com/image/2596c6306059524)
nnedi3_rpow2 (rfactor=2) (the leaves seem unreal after upscaling, look a little plastic)
http://thumbnails112.imagebam.com/30606/fd7f7a306059741.jpg (http://www.imagebam.com/image/fd7f7a306059741)
eedi3_rpow2 (rfactor=2) (the leaves seem fine)
http://thumbnails111.imagebam.com/30606/84f239306059984.jpg (http://www.imagebam.com/image/84f239306059984)

I donno why but it seems nnedi gives me sharper result than eedi and thinner edges, it's good for most of the image but not good for the leaves since they just look unreal, I wanna know if I can create a mask to mask out the leaves part and merge nnedi eedi results together (eedi for the leaves, and nnedi for the rest)
any ideas? ;)

[*.mp4 guy]

Function Twarp2(Clip c, int "thresh", int "wthresh", int "blurl", int "type", int "depth", int "chroma")
{

Thresh = Default(Thresh, 16)
wthresh = Default(wthresh, 127)
blurl = Default(blurl, 5)
type = Default(type, 1)
depth = Default(depth, 6)
chroma = Default(chroma, 1)

B1 = C.awarpsharp2(thresh=wthresh, blur=blurl, type=type, depth=depth, chroma=chroma)
Mt_LutXY(C, b1, " X Y - abs X Y - abs * X Y - * X Y - abs X Y - abs * "+string(thresh)+" + / 128 +", u=1, v=1)

Mt_AddDiff(last, b1, u=2, v=2)
}

blackmanresize(width*2, height*2, taps=10)
twarp2()

[feisty2]

glad that you show me your idea but how is this twarp2 function related to my problem ?

[jpsdr]

Try : nnedi3_rpow2(rfactor=2,nsize=0,nns=3,qual=2), and be sure to use v0.9.4.
Don't sure it may improve, but just in case. These are the paramaters i'm using.

[feisty2]

I tricked all nnedi3 parameters but leaves part seems no difference..:( just this plastic look, eedi looks fine, I wanna merge eedi and nnedi together but donno how to create a mask only including the leaves part

[poisondeathray]

eedi3 migth look better in some areas because it's leaving more artifacts and aliasing, or "false" details. nnedi3 might be "too clean" (anti aliasing too strong) hence the plasticky look. One way people deal with this is add grain for a textured look (of course this increases bitrate requirements)

Is there camera movement in this shot ? if this is a static shot, you can make a mask in your image editor

Otherwise I don't see any "easy" way of making a decent mask - there are no characteristics that separate the leaves very well from other parts of the picture (e.g you can't create good separation by saturation , Y' , hue - those are the most common - because there is too much overlap in characteristics with surrounding textures)

[feisty2]

is a complexity mask gonna work?
I noticed these leaves stuffs are high complexity areas