Port of NNEDI under new v2.6 AVS API

[jpsdr]

There is a missing dll:

32 bit W7 Core2 SSE4.2:
Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/nnedi3.dll'. Platform returned code 126:

Dependencies that couldn't be loaded:
libiomp5md.dll


Install the Intel Redistribuable as said in the ReadMe here (https://software.intel.com/sites/default/files/managed/05/a4/ww_icl_redist_msi_2016.4.246.zip).

It's working fine, nice, my threadpool is finaly good... ;)

[pinterf]

Install the Intel Redistribuable as said in the ReadMe here (https://software.intel.com/sites/default/files/managed/05/a4/ww_icl_redist_msi_2016.4.246.zip).

It's working fine, nice, my threadpool is finaly good... ;)
Thanks, stupid (read)me :)
Yesterday I was working on the prefetch problem (I was glad that it was finally easily reproducible), but could not fix it so far.

[jpsdr]

On what release version prefetch is still working properly ? I've made some little modifications on my threadpool, which theoricaly shouldn't have effect on actual situation (but everyone know the "theoricaly" and "shouldn't" ;) ), and i would like to be able to test.

[pinterf]

I will send you soon a link for a build, that seem to be OK.

Question:
Why are the filters specified as CACHE_DONT_CACHE_ME?

[jpsdr]

Are you talking about this :

int __stdcall nnedi3::SetCacheHints(int cachehints,int frame_range)
{
switch (cachehints)
{
case CACHE_DONT_CACHE_ME:
return 1;
case CACHE_GET_MTMODE:
return MT_MULTI_INSTANCE;
default:
return 0;
}
}


I've got this exemple from Ultim, it said it's the proper way to define the filter mode (and to not use SetFilterMode).
Now, i've just copied the code. I have no idea what the CACHE_DONT_CACHE_ME line is doing, neither why the exemple code of Ultim was like this. If you said that it's better to remove this line, i will.
(I'm not at home yet, so can't test the build yet).

[pinterf]

Remove it.
It's mainly for those filters that somehow manipulate the order or frames and do not filter the video itself.
(e.g. Trim, Reverse, SelectEvery, SeparateFields, AssumeFPS, DeleteFrame, DuplicateFrame, Loop, KillAudio)
If the frame is not cached, it will be recalculated for each getframe request if needed again. So removing that hint from the plugin will make the scripts faster (e.g. if a filter is looking back N frames)

[jpsdr]

New version. Issue with prefetch is solved, and you can also use with this version the last version of my internaly resampler filters, and you can also use prefetch with all of these...!

[real.finder]

Install the Intel Redistribuable as said in the ReadMe here (https://software.intel.com/sites/default/files/managed/05/a4/ww_icl_redist_msi_2016.4.246.zip).

It's working fine, nice, my threadpool is finaly good... ;)

test it with both NNEDI3 and ResampleMT Release_Intel_XP_Core2_SSE4.2 in winxp in VirtualBox and not work

http://i.imgur.com/rGFIYai.png

and before that I test in windows server 2008 r2 and not work too, the NNEDI3_v0_9_4_24 (Release_Intel_XP_Core2_SSE4.2) work fine

for ResampleMT I currently use Release_XP and work fine

[jpsdr]

Do you have up to date redistribuables installed ? 14.0.24215 for VS2015 and 2016.4.246 for Intel version ?

[real.finder]

Do you have up to date redistribuables installed ? 14.0.24215 for VS2015 and 2016.4.246 for Intel version ?

yes, I have them

[jpsdr]

No idea for now... :(

[jpsdr]

New version, see first post.

[jpsdr]

New version, see first post.

[real.finder]

Install the Intel Redistribuable as said in the ReadMe here (https://software.intel.com/sites/default/files/managed/05/a4/ww_icl_redist_msi_2016.4.246.zip).

It's working fine, nice, my threadpool is finaly good... ;)

I do it without it :rolleyes:

http://www.mediafire.com/file/3d536zh43m193po/icl.rar

work fine now in xp and windows server 2008 r2 ;)

[jpsdr]

work fine now in xp and windows server 2008 r2 ;)
Good to know.
The only thing i can see wich may has solved the issue is building with /MD instead of /MT...

[real.finder]

Good to know.
The only thing i can see wich may has solved the issue is building with /MD instead of /MT...
I did some changed now in the icl.rar

maybe, but I think intel dropped xp

in the first time I use libiomp5md.dll from Update 1 with svml_dispmd.dll from Update 4, but I feel there is something wrong with this mix, so I change 32 folder content now to xp and no xp

[Groucho2004]

I did some changed now in the icl.rar

maybe, but I think intel dropped xp

in the first time I use libiomp5md.dll from Update 1 with svml_dispmd.dll from Update 4, but I feel there is something wrong with this mix, so I change 32 folder content now to xp and no xp
It's a bit of a mystery that jpsdr creates binaries that need the MS and/or Intel support libraries but neither provides download links nor re-distributes them.

[jpsdr]

but neither provides download links nor re-distributes them.
I know by experience that links may change and/or disapear, making the link provided useless. Also binaries can evolved, for VS2015, we are now at leat at the second or third version of redistribuable only for the Update 3.
So, i rather put a comment on the readme saying to the people to get the redistribuable, this way they get the last up to date, and there is no "not working" link provided.

[Groucho2004]

I know by experience that links may change and/or disapear, making the link provided useless. Also binaries can evolved, for VS2015, we are now at leat at the second or third version of redistribuable only for the Update 3.
So, i rather put a comment on the readme saying to the people to get the redistribuable, this way they get the last up to date, and there is no "not working" link provided.
Fair enough. It's up to you.
By the way, the DLL from Release_Intel_XP_Core2_SSE4.2" does not work on XP. I installed the latest Intel redist package but Dependency Walker reveals that LIBIOMP5MD.DLL is looking for a function in kernel32.dll that does not exist on XP.

[jpsdr]

This seems to confirm what real.finder said.
Several possibility :
- Intel Droped XP support with last Intel redist release (strange, because in the IDE of ICL you can still choose "v140_xp").
- Intel messed up the last Intel redist release.
- There still XP support, but there is another "hidden" compiler (or linker) command to add in the build, as there is in VS2015 to still have an XP compatible build.
For now, i must said that i don't know. The workaround is to use the MS version under XP.

[real.finder]

By the way, the DLL from Release_Intel_XP_Core2_SSE4.2" does not work on XP. I installed the latest Intel redist package but Dependency Walker reveals that LIBIOMP5MD.DLL is looking for a function in kernel32.dll that does not exist on XP.

it's work here if you use this trick http://www.mediafire.com/file/5rp8jtv78otxxku/icl+-+xp.rar

tested in winxp sp3 32bit in VirtualBox

the last Intel redist package is the problem, use this https://software.intel.com/sites/default/files/managed/b4/98/ww_icl_redist_msi_2016.1.146.zip

[FranceBB]

Thanks for the "trick".
With that, it works just fine.
Tested a few minutes ago with:

nnedi3_resize16(target_width=1280, target_height=720, mixed=true, thr=1.0, elast=1.5, nns=4, qual=2, etype=0, pscrn=4, threads=0, kernel_d="Spline", kernel_u="Spline", taps=12, f_d=1.0, f_u=2.0, sharp=0)

to downscale from 1080p to 720p.
Thanks for the update! ;)

[jpsdr]

I found something odd in the nnedi3 code, and i think there is an error :

for (int y=0; y<ydia; ++y)
{
const uint8_t *srcpT = srcp+y_stride;

for (int x=0; x<xdia; ++x, ++input)
{
sum += srcpT[x];
sumsq += srcpT[x]*srcpT[x];
input[0] = srcpT[x];
}
y_stride+=stride2;
}
const float scale = 1.0f/(float)(xdia*ydia);
mstd[0] = sum*scale;
mstd[1] = sumsq*scale-mstd[0]*mstd[0];

If think we should have this instead :

mstd[1] = sumsq*scale*scale-mstd[0]*mstd[0];

Anyone is welcomed to comment.

[feisty2]

well, you could have a try and see if it still works..
only tritical will ever know the exact answer

[jpsdr]

It's too bad he's not on doom9 anymore... :(

[ajp_anton]

Not knowing exactly what that part is for, but yeah, it sure looks odd.

Breaking it down, if
- sum is just the sum of srcpT's, whatever those are.
- sumsq is the sum of the squared srcpT's.
- "sqsum" is the square of the sum (introducing my own variable).
then
- mstd[1] = (sumsq - sqsum*scale)*scale
which looks weirdly unbalanced. Either
- mstd[1] = (sumsq - sqsum)*scale
or
- mstd[1] = (sumsq - sqsum)*scale*scale
would look better. I guess it's the latter (same as your suggested edit) because mstd[0] already has one scale, so squaring that has two.

Edit:
Then again, squaring scale is also weird, becase it's basically (the number of elements in the sum)^-1, so it's a normalization factor. Maybe it's supposed to be (sumsq - sqsum)*scale ?
Like feisty said, try and see the results.

[Wilbert]

There's definitely something wrong, but you should look at entire source to figure out how to correct it.

Sadly any documentation in the source is missing. Here is my take. Disclaimer i understand nothing of the source.

Most of the fun seems to be happing in the function evalFunc_2 in nnedi3.cpp. The code:


void evalFunc_2(void *ps)
{
...
const int qual = pss->qual;
const float scale = 1.0f/(float)qual;
void (*extract)(const uint8_t*,const int,const int,const int,float*,float*);
void (*wae5)(const float*,const int,float*);

if (opt==1) wae5=weightedAvgElliottMul5_m16_C;
else wae5=weightedAvgElliottMul5_m16_SSE2;
...
if (fapprox&2) // use int16 dot products
{
if (opt==1) extract=extract_m8_i16_C;
else extract=extract_m8_i16_SSE2;
...
}
else // use float dot products
{
if (opt==1) extract=extract_m8_C;
else extract=extract_m8_SSE2;
...
}
...
extract(srcpp+x,src_pitch,xdia,ydia,mstd,input);
...
wae5(temp,nns,mstd);
...
if (opt>1) castScale_SSE(mstd,&scale,dstp+x);
else dstp[x]=min(max((int)(mstd[3]*scale+0.5f),0),255);
...
}

Looking at the last line, it implies that mstd[3] and the destination pixels differ a factor scale (since dstp[x]=mstd[3]*scale, removing the rounding).
castScale_SSE is defined nnedi3_asm.asm, but i don't know how to read asm.

The function weightedAvgElliottMul5_m16_C which is called in evalFunc_2 (and is set to wae5) gives another clue:

void weightedAvgElliottMul5_m16_C(const float *w,const int n,float *mstd)
{
...
if (wsum>min_weight_sum[0]) mstd[3]+=((5.0f*vsum)/wsum)*mstd[1]+mstd[0];
else mstd[3]+=mstd[0];
}

This implies that mstd[3], mstd[1] and mst[0] should be of the same scale.

Later on in the code, extract_m8_i16_C/extract_m8_i16_SSE2/extract_m8_C/extract_m8_SSE2, is set to extract. The function extract is called as

extract(srcpp+x,src_pitch,xdia,ydia,mstd,input);

Here mstd is defined. jspdr pasted some code of the function extract_m8_C, but the issue is there in all of these four functions. In extract_m8_C we see

void extract_m8_C(const uint8_t *srcp,const int stride,const int xdia,const int ydia,float *mstd,float *input)
{
...
const float scale = 1.0f/(float)(xdia*ydia);

mstd[0] = sum*scale;
mstd[1] = sumsq*scale-mstd[0]*mstd[0];
mstd[3] = 0.0f;
if (mstd[1]<=FLT_EPSILON) mstd[1]=mstd[2]=0.0f;
else
{
mstd[1]=sqrtf(mstd[1]);
mstd[2]=1.0f/mstd[1];
}
...
}

mstd[0] and sum (the source pixels) differ a factor scale which is consistent with the above. That is, if the value of scale in extract_m8_C is the same as scale in evalFunc_2. I have no idea if that's the case.
If we change 'mstd[1] = sumsq*scale-mstd[0]*mstd[0];' to 'mstd[1] = sumsq*scale*scale-mstd[0]*mstd[0];', it implies that mstd[1] and mstd[0] differ a factor scale, but mstd[1] is overwritten by its square root later on, so 'mstd[1]=sqrtf(mstd[1]);'. So now mstd[1] and mstd[0] have the same scale which is consistent with the above.
So you need to change that in all four functions.

What i don't understand what mstd[2] is supposed to do. It has scale^(-1) compared to mstd[1]. I don't see where mstd[2] is used, and thus if its scaling is correct.

[Wilbert]

mmm scale in evalFunc_2 is set to '1.0f/(float)qual;' with qual being an input parameter (being 1 or 2). While scale in extract_m8_C is equal to '1.0f/(float)(xdia*ydia);'.

qual doesn't seem equal to xdia*ydia to me?? xdia and ydia are set by

pssInfo[i].xdia = xdiaTable[nsize];
pssInfo[i].ydia = ydiaTable[nsize];

and these tables by (see header file):

const int xdiaTable[NUM_NSIZE] = {8,16,32,48,8,16,32};
const int ydiaTable[NUM_NSIZE] = {6,6,6,6,4,4,4};

[jpsdr]

Finaly, after viewing things in statistic way, it's good. mstd : probably Mean STandard Deviation.
mstd[0] is mean, mstd[1] is mean standard deviation, which is the square root of : mean of the sum of the squares, less the square of the mean.

Sorry, my mistake.

[Wilbert]

Finaly, after viewing things in statistic way, it's good. mstd : probably Mean STandard Deviation.
mstd[0] is mean, mstd[1] is mean standard deviation, which is the square root of : mean of the sum of the squares, less the square of the mean.
Yes indeed.

Your post is a bit cryptic. I think you are right that it should be

mstd[1] = sumsq*scale*scale-mstd[0]*mstd[0];


But i also think that the scale variables in evalFunc_2 and in the extract functions should be the same. I don't understand why they are different.

[jpsdr]

Again error from my side, the mean standard deviations is not what i've said after checking (my memory was not exactly right). We are not far, but it's not exactly what is calculated here.
But, what is done here is the mean of the squares less the square of the mean, and viewing like this, it can somehow make sense. So, maybe the formula is correct.

[Wilbert]

I give up. Leave the bugs in.
But, what is done here is the mean of the squares less the square of the mean
This is called the variance, and if you take the square of it you will get the standard deviation. Thus

VAR[X] = E[(X-E[X])^2] = E[x^2]-E[X^2], SD[X] = sqrt(VAR[X])

[feisty2]

I give up. Leave the bugs in.

This is called the variance, and if you take the square of it you will get the standard deviation. Thus

VAR[X] = E[(X-E[X])^2] = E[x^2]-E[X^2], SD[X] = sqrt(VAR[X])

should be E(x^2) - E(x)^2

EDIT: Var(x) = E((x-E(x))^2) = E(x^2 - 2xE(x) + E(x)^2) = E(x^2) - 2E(x)E(x) + E(x)^2 = E(x^2) - E(x)^2

[jpsdr]

So, finaly there is probably no bug, sumsq*scale-mstd[0]*mstd[0] produce the variance.
E(x^2)=sumsq*scale
E(x)^2=mstd[0]*mstd[0]
No...?

Still not been able to get 16bits working, and i can't figure out where it's going wrong.... :angry:

[Wilbert]

So, finaly there is probably no bug, sumsq*scale-mstd[0]*mstd[0] produce the variance.
E(x^2)=sumsq*scale
E(x)^2=mstd[0]*mstd[0]
No...?

Still not been able to get 16bits working, and i can't figure out where it's going wrong.... :angry:
E(x^2)=sumsq*scale^2 as i see it, but i guess i can't convince anyone.

Anyway. This scale factor is 1 by default (= qual input parameter). Could you make some screenshots voor qual=1 and qual=2 and compare them?

[StainlessS]

Could you make some screenshots voor qual=1 and qual=2 and compare them?

This is an English only forum, please don't post in foreign language here, I don't want to have to draw an administrators attention to this. Thank you for your compliance. :)

Merry Xmas Wilbert et al. :) [Latin dont count as a foreign language as only dead Romans speak it + a few Swiss Romansch nearly Roman speakers [bout 10,000 I believe]]

[jpsdr]

After a bloody and painfull struggle, i've been able to make the 16bits working.
Can someone explain to me why this is working :

const uint8_t *srcp = pss->srcp[b];
const uint8_t *srcpp = srcp-(ydia-1)*src_pitch-xdiad2m1;

and why this is not (at least with VS2015 community) :

const uint8_t *srcp = pss->srcp[b];
const uint8_t *srcpp = srcp-((ydia-1)*src_pitch-xdiad2m1);

???????????

Thanks again again to feisty2 for the code, it was very usefull, especialy for the init part and weight calcul adjustment.
And future thanks also for the part i'll begin to work in : the ASM ! The code will be helpfull.

[Groucho2004]

Can someone explain to me why this is working :

const uint8_t *srcp = pss->srcp;
const uint8_t *srcpp = srcp-(ydia-1)*src_pitch-xdiad2m1;

and why this is not (at least with VS2015 community) :

const uint8_t *srcp = pss->srcp[b];
const uint8_t *srcpp = srcp-[B]((ydia-1)*src_pitch-xdiad2m1);

???????????
Because the additional braces in the second statement change the precedence in which the variables are evaluated.

[jpsdr]

Argh... Back home too late to delete my stupid question after i've realised it... :(

[pinterf]

Great news, I suppose the hard thing was having uint16_t instead of a byte, does it automatically work for e.g. 10 bit videos? (Ideally all filters that work for 16 bits should also support 10, 12 and 14 bit videos)

[jpsdr]

Yes, thanks to code of the vapoursynth part done with trictical apparently, the hardest part (adjust the value of the weight coefficients) was allready done, and the values are adjusted according bit depth (and the float part is also done for the weight coefficients). The code automaticaly works with 10,12,14 and 16, with the fiew tests i've made.

My most struggle was with my stupid mistake from my previous post, i was stuck and i've been loosing hairs for several days because of it... Big step for me, but still work in progress. Once the pure C part will be done and fully tested, the next chalange: the ASM.
But again, i've help from the vapoursynth code.

[pinterf]

Once the pure C part will be done and fully tested, the next chalange: the ASM.
But again, i've help from the vapoursynth code.
Yes, it became more and more sophisticated over time in VS, I've seen there nice FMA3 parts, I'm not saying that those lines are just a relaxing afternoon read :)

[real.finder]

hi jpsdr

YUY2 is broken in x64 http://forum.doom9.org/showpost.php?p=1791047&postcount=17

[feisty2]

yuy2 is bullshit, convert whatever you got to yv16.

[jackoneill]

Yes, thanks to code of the vapoursynth part done with trictical apparently, the hardest part (adjust the value of the weight coefficients) was allready done, and the values are adjusted according bit depth (and the float part is also done for the weight coefficients). The code automaticaly works with 10,12,14 and 16, with the fiew tests i've made.

My most struggle was with my stupid mistake from my previous post, i was stuck and i've been loosing hairs for several days because of it... Big step for me, but still work in progress. Once the pure C part will be done and fully tested, the next chalange: the ASM.
But again, i've help from the vapoursynth code.

No tritical, just logic plus trial and error.

[jpsdr]

@real.finder Thanks for report, i'll look into it after i've finished my actual work.
@jackoneill According the vapoursynth code (if i've understood properly), the same asm prescreener functions are used for both 8 and 16 (or values between) bits. But, my tests until now (if i've not made errors) show me they are not working for more than 8 bits. Are you sure they should realy work ?

Edit : Finaly, i took a quick look out of curiosity, because something working in x86 but not in x64, make 100% sure it's in the asm code. Finaly i've quickly fond something. If i've not seen it, it's because i've only tested nnedi3_rpow2 for now, and within it, YUY2 is splitted in planar mode, so nnedi3 was never directly called with YUY2 format in my tests.

[jackoneill]

@jackoneill According the vapoursynth code (if i've understood properly), the same asm prescreener functions are used for both 8 and 16 (or values between) bits. But, my tests until now (if i've not made errors) show me they are not working for more than 8 bits. Are you sure they should realy work ?


There are three prescreener functions:
1) computeNetwork0 - "old" prescreener (pscrn=1), float dot products. This one you can use for all bit depths, including 32 bit float, after you adjust the prescreener weights. It is slow.

2) computeNetwork0_i16 - "old" prescreener (pscrn=1), int16 dot products. This one uses pmaddwd, which means that it can be used as is only for 8..15 bit pixels. (You still need to adjust the prescreener weights.) It's supposed to be faster than the float version, but who knows.

3) computerNetwork0new - "new" prescreener (pscrn=2..4), int16 dot products. It has no float version. This one also uses pmaddwd, so it also can be used as is only for 8..15 bit pixels. This one makes nnedi3 considerably faster, because more pixels end up processed with the fast bicubic method.

If you want to process 16 bit pixels with the faster prescreener, you could cheat and shift the pixels by one bit.

[jpsdr]

For now, only the C version of computeNetwork0_i16 and computerNetwork0new produce correct result, the ASM version produce incorrect results with bits>8. I'll have to continue testing/investigating, but actualy, that's the result i have.

[jackoneill]

For now, only the C version of computeNetwork0_i16 and computerNetwork0new produce correct result, the ASM version produce incorrect results with bits>8. I'll have to continue testing/investigating, but actualy, that's the result i have.

Remember that the weights need to be arranged differently for the ASM functions. You can't just use opt=1 (C functions only) and plug in one ASM function for testing, or vice versa.

[jpsdr]

Yes i know, and i have to be sure of everything, that's why for now i'm still under testing/checking.
Float is i think working properly, i'm still working on the 9..16 bits part.