Function + comment = error

[AlanHK]

I found that if in any function definition I have a comment after the function name and before the body, it throws an error:

AVISource("Garden.avi",false)
zz()

Function zz(clip a) # useless function
{return a}
same for
Function zz(clip a)
# useless function
{return a}

The error is "expected "{"

Which was an unexpected error, at least for me.

[tebasuna51]

You are right.
Maybe a 2.58 issue?
With 2.57 was OK.

BTW this work:

Function zz(clip a) { # useless function
return a}

[Gavino]

Maybe a 2.58 issue?
With 2.57 was OK.
Are you sure? I tried with 2.57 and got exactly the same behaviour as 2.58.

Looks like this is a 'feature' of the tokeniser. Newlines are skipped if the next token is '{', but not when there is a comment present.

Using the new /* ... */ form of comment available in 2.58 does work, ie
Function zz(clip a) /* useless function */
{return a}
is accepted.

[Gavino]

It's not clear to me if this is intended behaviour or an oversight.
However, since newlines are allowed before a '{', it seems inconsistent not to allow a comment here too.

I believe a simple fix is to change the code that handles #-comments (in Tokenizer::NextToken(), tokenizer.cpp) from
// skip from # to end of line (comment)
while (*pc != 0 && *pc != '\n' && *pc != '\r')
pc++;
break;
to
// skip from # to end of line (comment)
while (*pc != 0 && *pc != '\n' && *pc != '\r')
pc++;
if (*pc == 0) break;
continue;
This would force the newline terminating the comment to be treated like any other newline.

As an added bonus, it would also allow a '\' continuation on the line following a comment, something not possible at the moment.
Two extensions for the price of one.

[Gavino]

Looking further at this, my suggested fix above would only be a partial solution, since it doesn't help when the comment (even a new-style one) is on a different line to the function header, eg
function zz(clip a)
/* some inane comment ... */
{ ...
A better approach is to do it in the parser rather than hacking the lexer with even more special cases, by changing the code in ScriptParser::ParseBlock() from
if (braced)
Expect('{');
to
if (braced) {
// allow newlines (and hence comments) before '{'
while (tokenizer.IsNewline())
tokenizer.NextToken();
Expect('{');
}
The code in Tokenizer::NextToken()
} else if (*pc == '{') {
break;
could then safely be removed. (probably! :))

The revised fix no longer extends to allow a '\' continuation on the line following a comment, but I'm not sure that is desirable anyway.