Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
5th March 2017, 22:12 | #1 | Link |
Registered User
Join Date: Mar 2007
Posts: 407
|
StrResolve: String Variables Resolving Plugin (for masktools expressions)
I was always bothered that especially masktools expr="..." strings were tedious to read and manage as soon as externally calculated variables are involved.
E.g. when gamma is a float variable like gamma = 1.2 and you want to use gamma in a mt_lut expression you need to write Code:
gamma = 1.2 "(( x/255 ) ^ " + gamma.string + ") * 255" \.mt_polish Here you'll find a string helper plugin StrResolve. It takes a string and parses words prefixed by a prefix string if they are existing AviSynth variables in the scope of the plugin. If so, it retrieves their value, converts them to string if neccessary and inserts this in the place of the word. Otherwise an error is issued. The benefit over the previous solution "...".ReplaceStr("gamma", gamma.string) is that it is universal, i.e. it recognizes all variables, regardless of their names, and needs only be called once in case of multiple variables. With the plugin, it's Code:
gamma = 1.2 expr = "(( x/255 ) ^ $gamma) * 255" \.StrResolve.mt_polish The documentation is in an example AVS file in the ZIP. v.0.5.0, follows the semver (semantic versioning) standard and other plugin writers are encouraged to adopt this, and copy the compile time formatter to automate yyyy/mm/dd-comparable compile time documentation. Also, it features an # inline comment removal and multi line->single line reformatting, to make masktools' expressions even more human readable. Finally, I could not resist and included a feature to not only insert values from variables, but also enter script snippets between curly brace twins: {{...}} which are executed with Eval("...") and the result is also inserted in the place of the code. This should enable most everything inside expression strings. Edit March 10, 2017: v0.5.0, as described above. Testers welcome, probably not yet bug free. Is the name still OK or are there better proposals? Last edited by martin53; 11th March 2017 at 21:09. |
5th March 2017, 22:35 | #2 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
just download and see there are no dll, just source code
__________________
See My Avisynth Stuff |
6th March 2017, 00:54 | #3 | Link |
Retried Guesser
Join Date: Jun 2012
Posts: 1,373
|
I embed variables in expressions like this...
Code:
s = "((x/255)*$a)^(1/$g)*255+$b" \ .StrReplace("$a", String(gain)) \ .StrReplace("$g", String(gamma)) \ .StrReplace("$b", String(bias)) \ .mt_polish |
6th March 2017, 19:05 | #4 | Link |
Registered User
Join Date: Mar 2007
Posts: 407
|
Currently the masktools reserved words are case insentitively not resolved.
It would be possible to e.g. not resolve x, but to check if X can be resolved, if that is preferred. That way, lowercase letters could be used to identify masktools placeholders and reserved words, uppercase letters to identify variables (or vice versa, but only one alternative). (EDIT 2017/03/10 no more true for current version) Last edited by martin53; 11th March 2017 at 21:12. |
6th March 2017, 19:24 | #5 | Link |
Excessively jovial fellow
Join Date: Jun 2004
Location: rude
Posts: 1,100
|
I'm probably a fool for asking this, but whyyyyyyy. Why, doom9 forums poster martin53? Why did you do it? Why did you rethink that which had no need of being rethunk? Why did you write code? Much like you shouldn't reinvent wheels (especially not square ones), you shouldn't solve solved problems. I have an old friend who is hurt and confused by your actions, and I think you should apologize to him. His name is sprintf, and I really don't think he's ever done anything to hurt you.
If you had just made a plugin that was nothing more than a thin wrapper around sprintf, he would have been good to you. He is old, but he's reliable! Many hundreds of millions - maybe even billions - of users have trusted him to do the right thing in the past. If you let him solve your problems, you could have had many nice things that people have enjoyed for many years, now. Formatting floats with arbitrary precision! Converting floats to integers! Re-ordering arguments! All kinds of nifty stuff! The language interpreter itself could handle mistyped variable names! You could separate data from code! You wouldn't even have had to do anything, my old buddy would have handled it all for you! Why did you have to betray him like this? (it's actually really weird that avisynth doesn't actually seem to have have sprintf in the language already - it's even exposed as a function in the public api. there's a dumb one-variable sprintf in the String() function but idgi, why would you even do that and not have the real thing) Last edited by TheFluff; 6th March 2017 at 19:30. |
6th March 2017, 20:02 | #6 | Link | |
Registered User
Join Date: Mar 2007
Posts: 407
|
Quote:
Eval("...") and GScript("...") directly work on interpreter level and I currently can't imagine a use case where it is useful to resolve a variable name before interpreting the surrounding script source text. (I have the hidden thought in my mind that the core functionality is transferred to mt_lut...()'s processing when it has no clue about the next token, and it was a finger exercise to not only post a suggestion, but give a fully working implementation of the proposal. But I don't want pinterf to just copy and include it: rather would I fork his Github and then make a pull request with it for the addition as another finger exercise ) |
|
6th March 2017, 20:25 | #7 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
It kinda looks sexy to have arg names embedded in the string.
I really do dislike the string concatenation stuff and the String() use as usually used in AVS, I think I did look at fluffy suggested stuff in API and did not like the 4KB limitation, so RT_String() dont use it (neither do RT_SubTitle nor RT_WriteFile nor RT_DebugF), but for most small case use I guess the fluffy one is correct, and the API would quite suffice.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
6th March 2017, 20:30 | #8 | Link |
Registered User
Join Date: Mar 2007
Posts: 407
|
RT_String() vs. StrResolve() functionalities are very different. Taking Raffriff42's example
"((x/255*gain)^(1/gamma)*255+bias)", that would make the doubtlessly nice approach Code:
gain = 1.0 gamma = 1.0 bias = 1.0 expr = "((x/255*%f)^(1/%f)*255+%f)" \.RT_String(gain, gamma, bias) Code:
gain = 1.0 gamma = 1.0 bias = 1.0 expr = "((x/255*gain)^(1/gamma)*255+bias)" \.StrResolve Btw. everyone is free to convert float variables into strings in advance. Then StrResolve takes the strings, of course. See AviSynth's String(float / int [, string format_string]) function. Just change gain = 1.0 to gain = (1.0).String("%.10f") but don't forget AviSynth stores in 32 bits! 420.3 is wrong in the 5th digit after the decimal point already. Last edited by martin53; 11th March 2017 at 21:13. |
6th March 2017, 21:57 | #9 | Link | ||
Excessively jovial fellow
Join Date: Jun 2004
Location: rude
Posts: 1,100
|
oookay, you asked for it.
Quote:
Did you read the part where I said the language interpreter could do this for you? Not only does it also consider global variables, it will do the right thing and immediately detect your bugs for you by throwing an exception if you pass a variable that doesn't exist. Have you people never heard of the concept of a "type error"? Speaking of type errors, your implementation also does the wrong thing silently with things that aren't strings. And what do you even mean by a "parser"? Quote:
If you want variable interpolation in strings, write Perl, for chrissakes, but not even Perl accepts random words as identifiers without a marker of some kind, and it has delimiters and escaping for this kind of thing. Really, if your goal is to make rpn with embedded variables readable, what you want is a templating engine that supports inline comments. Five people probably wrote one in Javascript this week - don't get left behind! If you don't want to be that hipster, an sprintf implementation that supports named variables (such things exist! mindblowing, huh!) would be an acceptable substitute. Last edited by TheFluff; 6th March 2017 at 22:01. |
||
6th March 2017, 23:52 | #10 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Martin53, I had not downloaded your source, when I had tried, I got a 'MediaFire is having Problems' or similar type message, downloaded now and had a peek.
Its a pleasure to see that you are such a tidy coder (so much tidier and less messy than that DavidHorman, lest wise that what he tells us, we never get to actually see if he tells true on that ). Anyways, the fluffy one again seems to have cast his evil spell, and perhaps rightly so (dont you just hate it when that happens). If you want to put out a mod of RT_String, you are welcome, perhaps you could improve and convert to use some of the M$ more secure (though less portable) library functions. I've ripped out the 'esc' arg, just allowed escape sequences in data strings which I'm quite sure that you would not want. There are likely some things that are not needed for MaskTools type use, (eg %X hex), take them out if you wish or no harm to leave in, the Sprintf thng is only really of use for devs in creating AVSValue strings for return to the user, not really suited for use by the user in a String() type function. Here supplied code allows parsing of the user supplied format string and args list. If you improve on it, I'm bound to steal some of it back off you. I Suggest the name FString(), but your call. Code:
AVSValue __cdecl RT_String(AVSValue args, void* user_data, IScriptEnvironment* env) { char *myName="RT_String: "; const char *s = args[0].AsString(); // text int arrsz = args[1].ArraySize(); enum { CHICKEN=64 }; // what size buffer we need ? int i,mem=strlen(s) + 1 + CHICKEN; for(i=0;i<arrsz;++i) { if(args[1][i].IsString()) { const char *st=args[1][i].AsString(); mem += strlen(st) + 1 + CHICKEN; } else { mem += 8 + CHICKEN; // no particular reason why so big, just chicken factor. } } char *pbuf = new char[(mem+1)*2]; if(pbuf==NULL) env->ThrowError("%sCannot allocate memory",myName); char *ptem=pbuf+(mem+1); // temp buffer const unsigned char * r= (const unsigned char *)s; char *p=pbuf; int c,ix=0; int t=0; // Parse text and insert variables while(c=*r) { if(c=='%') { ++r; if(*r=='\0') { *p++ ='%'; } else if(*r=='%') { *p++=*r++; // replace escaped double % with single } else { if(ix>=arrsz) { delete [] pbuf; env->ThrowError("%sExpecting data arg (%d)",myName,ix+1); } char *tp=ptem; *tp++='%'; if(*r=='-' || *r=='+' || *r=='0' || *r==' ' || *r=='#') // flags *tp++=*r++; if(*r=='*') { // int holds length t=args[1][ix].IsBool() ?1: \ args[1][ix].IsString() ?2: \ args[1][ix].IsInt() ?3: \ args[1][ix].IsFloat() ?4: \ 0; if(t!=3) { delete [] pbuf; env->ThrowError("%sUnsupported data type, Expecting Width as Int (%d)",myName,ix+1); } tp+=sprintf(tp,"%d",args[1][ix].AsInt()); // EDIT: FIXED: WAS IsInt ++r; // skip '*' ++ix; // next data } else { while(*r>='0' && *r<='9') { *tp++ = *r++; } } if(*r=='.') { *tp++ = *r++; // precision prefix if(*r=='*') { // int holds length t=args[1][ix].IsBool() ?1: \ args[1][ix].IsString() ?2: \ args[1][ix].IsInt() ?3: \ args[1][ix].IsFloat() ?4: \ 0; if(t!=3) { delete [] pbuf; env->ThrowError("%sUnsupported data type, Expecting Precision as Int (%d)",myName,ix+1); } tp+=sprintf(tp,"%d",args[1][ix].AsInt()); ++r; // skip '*' ++ix; // next data } else { while(*r>='0' && *r<='9') { *tp++ = *r++; } } } t=args[1][ix].IsBool() ?1: \ args[1][ix].IsString() ?2: \ args[1][ix].IsInt() ?3: \ args[1][ix].IsFloat() ?4: \ 0; // type if( (*r=='c' ) || (*r=='C' ) || // char as int (*r=='d' || *r=='i') || // int (*r=='o' || *r=='u' || *r=='x' || *r=='X')) { // unsigned int if(t!=3) { delete [] pbuf; env->ThrowError("%sType='%c', Expecting Int data (%d)",myName,*r,ix+1); } *tp++=*r++; *tp='\0'; p+=sprintf(p,ptem,args[1][ix].AsInt()); ++ix; // next data } else if(*r=='e' || *r=='E' || *r=='f' || *r=='g' || *r=='G') { // double if(t!=4&&t!=3) { delete [] pbuf; env->ThrowError("%sType='%c', Expecting Float (%d)",myName,*r,ix+1); } *tp++=*r++; *tp='\0'; p+=sprintf(p,ptem,args[1][ix].AsFloat()); ++ix; // next data } else if((*r=='s')||(*r=='S')) { // string if(t!=2&&t!=1) { delete [] pbuf; env->ThrowError("%sType='s', Expecting String (%d)",myName,ix+1); } *tp++=*r++; *tp='\0'; if(t==1) { // Bool p+=sprintf(p,ptem,args[1][ix].AsBool()?"True":"False"); } else { // String p+=sprintf(p,ptem,args[1][ix].AsString()); } ++ix; // next data } else { delete [] pbuf; env->ThrowError("%sUnknown format type '%c' (%d)",myName,*r,ix+1); // eg Clip } } } else if(c == '\\') { ++r; c=*r; // abfnrtv switch (c) { case '\0' : *p++='\\'; break; // copy single backslash at end of string case '\\' : *p++=*r++; break; // replace double backslash with single backslash case 'n' : ++r; *p++='\n'; break; case 'r' : ++r; *p++='\r'; break; case 't' : ++r; *p++='\t'; break; case 'v' : ++r; *p++='\v'; break; case 'f' : ++r; *p++='\f'; break; case 'b' : ++r; *p++='\b'; break; case 'a' : ++r; *p++='\a'; break; default : *p++='\\'; *p++=*r++; break; // anything else we copy backslash and whatever follows } } else { *p++=*r++; } } *p=0; // nul term if(ix<arrsz) { delete [] pbuf; env->ThrowError("%sUnexpected data arg (%d)",myName,ix+1); } AVSValue ret = env->SaveString(pbuf,p-pbuf); delete [] pbuf; return ret; } extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) { env->AddFunction("RT_String", "s.*",String, 0); return "`RT_String' RT_String plugin"; } EDIT: BUGFIX IN BLUE
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 7th June 2017 at 18:39. Reason: FIXED BUG |
7th March 2017, 05:01 | #11 | Link |
I'm Siri
Join Date: Oct 2012
Location: void
Posts: 2,633
|
It's weird tho, cuz I'm also a fan of the c++ programming language and somehow StainlessS' code always looks like machine code to me..
Code:
enum { CHICKEN = 64 }; Can't you just make it readable, make it c++ like Code:
constexpr auto chicken = 64; Edit: you people are responsible for all the c++ haters out there, it's all that weird abusing shit that scared lots of people away Last edited by feisty2; 7th March 2017 at 05:05. |
7th March 2017, 12:30 | #12 | Link |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,492
|
How about an option to only replace variables when they are prefixed with $? Or just allow an optional user-specified prefix.
I will probably code something like this into future versions of xyremap/pixel_rpn etc. Strangely I had need of exactly such a thing for the first time just two days ago, though I didn't see the opportunity for doing it in this way, so I had a lot of string(...)s. Last edited by wonkey_monkey; 7th March 2017 at 12:33. |
7th March 2017, 19:33 | #13 | Link | |
Registered User
Join Date: Mar 2007
Posts: 407
|
Quote:
Pls read the AVS file. - Selectable prefix char added, defaults to ':' (EDIT: default "$" in later versions) which I judge unobtrusive and is not yet used in conflicting manners in today's RPN. unfortunately I named the parameter 'trig' instead of the obvious 'prefix' And if the majority of users prefers '$' I'll change it. - additional switch 'strict' added to adapt to user's intention of behaviour. Last edited by martin53; 11th March 2017 at 21:15. |
|
7th March 2017, 20:22 | #14 | Link | |
Registered User
Join Date: Mar 2007
Posts: 407
|
Quote:
(1) though I originally had a 'clean' RPN string in my mind, similar to a script source text, and personally don't like script languages that require prefix characters for variables, there are indeed good reasons to vote for a prefix; mostly the fact that the writer's intention is known, and an error can be issued at all. (2) I don't see how the script language interpreter can be of help here; maybe I lack some knowledge. The env->GetVar() function does consider local and global variables. The plugin now throws an exception by default, too. In the specific case of masktools' expressions, this is of limited value, because masktools itself silently breaks RPNs it cannot interpret. Regarding sprintf() once more: I looked at a stackoverflow.com example with named placeholders. But that approach will always have two occurences of the varying value: the placeholder and the actual variable. For someone like me that's fine in a text string (my mind is able to process that), but I don't like it in a program language source text or in a complex math formula. There I prefer the direct occurence of surrogates, like the most famous x variable. Code:
y = sin(x) <-> %y = sin(%x), $y, $x <-> %f = sin(%f), $y, $x Last edited by martin53; 7th March 2017 at 20:44. |
|
7th March 2017, 20:34 | #15 | Link |
Registered User
Join Date: Jan 2014
Posts: 2,309
|
Discuss and clean up the idea, I was thinking about similar things, but in connection of the possible integration of GScriptclip in avs+ core, but without specifying the parameter list string in a separate parameter. A more general method is always welcomed.
|
8th March 2017, 07:58 | #16 | Link |
Registered User
Join Date: Mar 2007
Posts: 407
|
These 0x20...0x7e ASCII characters are currently not used in RPN formulas:
$,:;\_`{} $ is already used in some other programming languages as a prefix for variable names. In AviSynth, it is the hex number prefix. It looks like AviSynth hex numbers always allow a leading zero, even if additionally prepended. This would be a possible and intuitive exact distinction from names of variables. (current RPN interpreters can not interpret hex numbers, but they might learn to) : though not directly used in RPN, it is part of the ternary Elvis operator. "x<16?y:x" is a valid infix expression and collides with variable name x. With given infix formulas, StrResolve must be executed before mt_polish (or mt_polish must learn how to transform the expressions, which I don't prefer). Although it is possible to write "x<16?y: x" instead, the mandatory space character is an unelegant syntactical requirement. , and ; these characters typically are found at the end of expressions or separators, it feels unusual to use them as prefixes. \ could be found in path strings, which is currently not an issue with RPN, but maybe restricts the function otherwise. _ not nice because it is also a valid variable name character. Variable X outside the expression is _X inside, _X outside is __X inside etc. - confusing. ` is not yet used and does work. Is it present on many language keyboard layouts. One reason not to use it might be the fact that it seems to be last character that can be used as an escaping character one day, because it is yet unused. I think it is even already used for escaping or so in linux? {} Curly braces come as a pair. It looks too ugly to write {x for variable x, and also to write }x. I had thought of setting {x} as syntax, but prefer a single prefix, because it's shorter. Conclusion I think I shall set $ the standard prefix character and rename the parameter to prefix in the next plugin version. Objections welcome. Last edited by martin53; 8th March 2017 at 08:02. |
8th March 2017, 09:56 | #17 | Link | ||
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,492
|
Quote:
Quote:
I think $ is the obvious choice, and I'm not just saying that because I write a lot of PHP. |
||
8th March 2017, 15:23 | #18 | Link |
Retried Guesser
Join Date: Jun 2012
Posts: 1,373
|
It can help to use source-clip names in expressions too, especially for mt_lutxyz:
Code:
# http://avisynth.nl/index.php/Color_conversions # # G = Y - 2*(U-128)*(1-Kb)*Kb/Kg - 2*(V-128)*(1-Kr)*Kr/Kg # sG = ("$Y-(2*($U-128)*(1-$Kb)*$Kb/$Kg)-(2*($V-128)*(1-$Kr)*$Kr/$Kg)") \ .StrReplace("$Y", "x") \ .StrReplace("$U", "y") \ .StrReplace("$V", "z") \ .StrReplace("$Kr", sKr) [* 0.2126 for PC.709 *] \ .StrReplace("$Kg", sKg) [* 0.7152 *] \ .StrReplace("$Kb", sKb) [* 0.0722 *] \ .mt_polish G = mt_lutxyz(Y, U, V, sG) |
8th March 2017, 21:28 | #19 | Link | |
Excessively jovial fellow
Join Date: Jun 2004
Location: rude
Posts: 1,100
|
Quote:
Code:
value = doSomething(argument) If you on the other hand take input by inspecting the current state of the calling scope, you need to do this name and type checking on your own. I haven't looked at your latest version but you previously used this implementation: Code:
int error = -1; if (!isReservedWord(token)) error = env->GetVarDef(token, AVSValue(-1)).AsInt(); // returns -1 when var in token doesn't exist if (error == -1) { // the token is reserved or not a variable // append token as is to output string strncat_s(resolvedString, 4096, token, strlen(token)); } else { ... Last edited by TheFluff; 8th March 2017 at 22:27. |
|
8th March 2017, 22:16 | #20 | Link | |
Registered User
Join Date: Mar 2007
Posts: 407
|
Quote:
@ALL The described behaviour sounds weird, but I must confirm: StrResolve is currently unable to resolve variables with value -1 because although they exist, the API function GetVarDef() - remember, the function which is made to tell a plugin if a variable exists - lies. Well, if the script writer needs to enter the variables that are used in the infix or RPN formula as additional parameters to the plugin, there is almost no ergonomic benefit left. Let's see if at least AviSynth+ can be improved here etc. EDIT: This was clarified later: GetVarDef() isn't made to tell if a variable exists, but avoid an exception if it does not exist. Still, by two calls with different defaults, the plugin can make a decision if the variable really exists (both calls return the same value). Different topic: I'm also a little concerned about the general version control (or better, its complete absence) in current worldwide spread plugins and scripts, and did improve the VersionInfo function to a state where I think I can suggest other plugin writers to copy it and use it as the least standard. Published scripts should then get the version info and string-compare it to a literal minimum version. The string has a free 1st part (for authors who love version 0.0.0.01.alpha), proposedly terminated by a space character, then a YYYY/MM/YY hh:mm:ss compile time stamp which is calculated automatically for those autors typically forgetting to increment the manually maintained version string. I encourage script authors do do such version compare in scripts they publish for others, to enhance the overall user experience with AviSynth. Last edited by martin53; 10th March 2017 at 21:20. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|