AviSynth+ thread Vol.2

[real.finder]

dude, deciphering assembly is HARD

how SEt, tp7 and jackoneill did it then?

anyway even if you got the source code I think it don't has C code just like most old plugins

[feisty2]

deciphering a few clearly outlined functions written in assembly with possibly also comments is not the same as blind-deciphering an executable... especially if the binary has been heavily optimized or involves some weird shit like C++ name mangling

[videoh]

Instead of all that reversing nonsense, you have to re-implement the algorithms. What's the big deal?

[FranceBB]

Instead of all that reversing nonsense, you have to re-implement the algorithms.

Re-implementing the algorithm, makes sense.

there are always reverse engineering option

Out of pure curiosity, you meant using things like IDA Pro and start from there? I'm not an avs developer, but I can tell you that staring at several lines of disassembled code totally undocumented is way beyond my understanding. (And I believe it would be a pain in the b... for many experienced developers as well).

dude, deciphering assembly is HARD

I'm always surprised by how people in the past used to rely on assembly for everything and were so good in it while I can't really wrap my mind around that as I generally lose track of the main picture after just a few lines (i.e I might understand what a single block of code is doing, but I fail to understand what the program as a whole is doing... :( )

(I still remember people of the David Brailsford (https://www.nottingham.ac.uk/news/expertiseguide/computer-science-/professor-david-brailsford-.aspx) era carrying mugs with "Real man code in Assembler" before retiring :( )

[feisty2]

"Real man code in Assembler"

(certain) engineers code in assembly, mathematicians code in lisp, scientists code in... who knows

anyways, it has nothing to do with "real man"

[real.finder]

Out of pure curiosity, you meant using things like IDA Pro and start from there? I'm not an avs developer, but I can tell you that staring at several lines of disassembled code totally undocumented is way beyond my understanding. (And I believe it would be a pain in the b... for many experienced developers as well).


I am not a programmer, but I see tp7 said this back then in IRC (about sangnom2)

also it's what SEt did in awarpsharp2

anyway, tp7 got the source code of Checkmate from Prunedtree as I said, so I think any developer can do ask Prunedtree for the code like https://forum.doom9.org/showthread.php?p=1792960#post1792960 and https://forum.doom9.org/showthread.php?p=1825279#post1825279

[Groucho2004]

(certain) engineers code in assembly, mathematicians code in lisp, scientists code in... who knows

anyways, it has nothing to do with "real man"It used to be a joke a long time ago (or maybe still is?), particularly when C# was introduced (as far as I recall).

Real men code in C++
No, real men code in C
No, real men code in assembler
No, real men punch in hex code on numeric keyboards
etc...

By the way, this is a piece of art:

https://i.postimg.cc/90sdNdR8/Dvrm3be-Uw-AAMCpg.jpg

[FranceBB]

By the way, this is a piece of art

Uhhh! Hexadecimal with a 7 Segments display! :D
I studied those in my Digital Electronics class!
This is what I came up with to count in Hex (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F) in a 7 Segments Display:

https://scontent-lhr8-1.xx.fbcdn.net/v/t1.0-9/s960x960/89858123_3065862253445594_4431349507128557568_o.jpg?_nc_cat=102&_nc_sid=e007fa&_nc_ohc=3rIGxBUhJ6kAX_cSlds&_nc_ht=scontent-lhr8-1.xx&_nc_tp=7&oh=d5f5e6f724292570533ae45ebc24f162&oe=5EDB51E6
https://scontent-lht6-1.xx.fbcdn.net/v/t1.0-9/s960x960/89872085_3065861016779051_1406615868342796288_o.jpg?_nc_cat=105&_nc_sid=e007fa&_nc_ohc=ZnwebgBMjEIAX-jEwla&_nc_ht=scontent-lht6-1.xx&_nc_tp=7&oh=9194f7524d44e3566ec534a14ef87d84&oe=5EDB9DC5



I studied a lot for that exam and I was sure I was going to kill it, but unfortunately things didn't go as planned I've got 23 out of 30 :(
I was so upset that day... :(
(but I accepted the grade of course, I have never refused one, that's my "mantra" xD)
That was back when I was studying for my bachelor degree...
Will I ever get my master degree? Who knows...

[Groucho2004]

Uhhh! Hexadecimal with a 7 Segments display! :D
I studied those in my Digital Electronics class!Cool, a fellow electronics engineer. How old are you, if I may ask?

[feisty2]

Will I ever get my master degree? Who knows...

masters are mostly cash-cow programs, go for a PhD program instead if ur interested in grad schools

[Groucho2004]

feisty, I only just noticed that you moved from CA to RI, how are you handling the temperature difference?

[feisty2]

I haven't, I changed my location in advance cuz I will begin my graduate study at one of the institutions on college hill, I ain't tellin which one

[Rumbah]

masters are mostly cash-cow programs, go for a PhD program instead if ur interested in grad schools

There is no PhD without a master first in Germany.

[FranceBB]

Cool, a fellow electronics engineer.

Actually, it was supposed to be Computer Science Engineering but when I got there I found out that we had many exams in common with those studying Electrical Engineering. I asked one of the professors why and he said "You see, Computer Science Engineering didn't exist years ago and it was Electrical Engineering" so years passed but some universities are very conservative in their courses and they kept Electrical Engineering and Computer Science Engineering closely bonded together. I had exams on the whole analog electronic part of which the first exam was huge as it covered everything from where physics II left with Induction Motors, Solar Panels etc, then I had the whole Control System part with Bode Plots and an introduction to Digital Electronics, then the whole part about Digital Electronics from the basics up to FPGA. The idea behind that was to teach us how components like CPUs, RAM etc work at a very deep level from an electrical point of view so that we could design a system. The thing is that although I learned many interesting things from linear algebra to multivariable calculus, to fluid dynamics and a lot of electrical engineering stuff, I never learned how to code properly; as a matter of fact, professors were very keen to make us code using programming languages that I'll never use in real life ('cause I'm an encoder) like Verilog and VHDL for hardware-programming or MATLAB for other kind of things. When I enrolled at University in a Computer Science Engineering class I thought things were going to be very different...


How old are you, if I may ask?

My age has never been a secret, in fact it's also in my profile here on Doom9. :) (I'm 26 by the way; I'm probably younger than you think).


masters are mostly cash-cow programs, go for a PhD program instead if ur interested in grad schools

I know but I can't get a PhD without a master degree; Anyway, I've seen the cost of university in the U.S and it's insane... luckily in Europe they cost way less as most of them are public.

CA

I've been in California in 2013 and I stayed there for three months, during summer. I was in Claremont, close to LA, in a University Campus where I improved my English a lot. Years later I tried to go back there to work in the U.S but VISA are a nightmare and I only got a student visa which they were willing to renew if I wanted as they said that if I went to university there I had many more chances to get a working visa and eventually a Green Card, however I had no money to pay for my education without working and I didn't really want to get a student loan as I know that some people are still paying their debs years after leaving university, so I flew to the UK ('cause I thought "what's the closest English-speaking country I can move to without a Visa? - thank you EU -") but then I heard of a company that was looking for an encoder in Berlin, I applied and I've got the job, so I moved to Röntgental (which is close to Berlin, but smaller and cheaper). After some time, I got transferred to Sky and "the rest is history". :)

[MeteorRain]

I hate schooling so I abandoned the idea to go for a PhD.

(Should a real man be coding on a mechanical computer?)

[StainlessS]

They had (some years back) the finally completed analytical engine in the sciece museum, London, Kennsington I think.
Apparently it works.

As mechanical a computer as you are likely to find. [The Charlie Babage thiing]

EDIT:
Whatever happened to the old mechanical 'sliderule' calculator type thing, where you stuck pegs in holes [or similar]
and turned a handle and got an answer, handheld devices, probably not too disimilar to the Babage thiing.
[I think I had one as a child, or at least seen one]

EDIT: "or similar", actually a metal slide thing, rather than peg, I think.

EDIT:
I remembered. My auntie [bout 18 months older than me] had one of these when I was about 10 YO.
Put the pen like nib in saw tooth slider thing, and dial in the numbers.
Pull UP/OUT, the cross bar whatsit on top of device, and hey presto, the answer.
https://i.postimg.cc/KRf246vs/Vintage-Fedtro-Precision-Mechanical-Pocket-Calculator-Made-In-Japan-Sold-For-69-Cents-Circa-1966.jpg (https://postimg.cc/KRf246vs)
https://i.postimg.cc/v1dkzXB6/calc.jpg (https://postimg.cc/v1dkzXB6)

[LigH]

I remember how I entertained my college class by tracing the self-modifying code of the EICAR test virus in DEBUG.COM ... okay. Back to AviSynth+ please?

[tormento]

Real men code AVS+ plugins.

So go back to work. :)

[Dogway]

and there are some filters seems has no VS ports

1 - VariableBlur (it was planed by tp7 (https://forum.doom9.org/showthread.php?p=1658610&highlight=VariableBlur#post1658610))
2 - frfun7 (useful for Dot Crawl Removal, used in DDComb)
3 - GradFun2db (maybe it can be replaced with f3kdb?)
4 - LGhost

Fortunately recently some of the plugins I used more frequently were updated (SSIM, SubtitleEx, AvsInpaint), still miss a few of them, maybe you can add them to a spreadsheet:
JpegSource
SoundOut
VScope

[pinterf]

Things that happened lately.

AviSynth+

Last week I have successfully applied the 'old' type+ and type* syntax on _named_ array function definitions. e.g. [colors]f+
It was not straightforward because there are user defined functions, function objects, parameter type and name matching logic. So it was much harder task than I have anticipated.
Future plugins using this parameter syntax will fail to load on older Avisynth hosts, but if someone would use array and this syntax then they must use AddFunction dynamically.

There are problems with those CPP v2.5 'baked code' interface plugins which are using "Invoke", latest example was GRunT for avs 2.58. I was trying to find a solution if providing a special ancient ScriptEnvironment for them is feasible. But I failed, way too difficult and messes up the internals of the current core.

I was experimenting with putting 1D and 2D lut into the core - using Expr - but unfortunately it won't be a couple of hours' work. I postponed the project.

Still considering on what other things would be nice to appear in the extended IScriptEnvironment but I think I'll make a feature freeze on it soon. Not to mention of documenting all new features.

Plugins

TDeint.
You know how angry I am at all 2.5 plugins. So I met TDeint again some weeks ago.
TDeint did not have a proper stable x64 port and it was a 2.5 CPP plugin. I started to work on it and sorry, I cannot stop the project now. I'm getting into deeper and deeper in the modifications which are all have to be done to have a proper source again.
TDeint and TIVTC have many-many common parts in source, which are not easy to detect. Piles of hundred line copy pastes with a minor difference (a part of them is because there existed no templates in C++ at that time tritical authored it), thousand lines of inline hand optimized asm code without C.

I worked many labour-weeks on TIVTC three years ago and had fed up a bit for this reason:) It's a never-ending story.
Three years ago I was optimistic and put up the actual TDeint on github together with TIVTC (iin case of I feel like working on it) but I didn't touch it.
Until last week.

Haha, you mentioned a week ago that TDeint has no high bit depth port? It didn't even support Avisynth 2.6 basic colorspaces. So all these issues (plus a ping on github) forced me to look at it again.
Lately YV16-YV24-YV411 was added. Today - not released but you can see it on git - I have finished the 10-16 bit port plus greyscale. I'm seeing the end of the tunnel and turned to TIVTC again and I think I won't stop until it's ready 16 bits.

This is a much better entertainment then solving crossword puzzles isn't it, but is a bit time consuming. RescueMyTime reported 300+ hours for the last three months (plus my official work), I'm so glad that I'm saving two hours per day on commute :) thanks to this stupid covid.

(Out of couriosity, do other developers watch films, videos or read books?

Nevertheless I'm just moaning :) So what about chikuzen's plugins (which have to be rebuilt for next Avs+?) Will they be maintained in the future by him? If not, what is order of their importance?

[real.finder]

Nevertheless I'm just moaning :) So what about chikuzen's plugins (which have to be rebuilt for next Avs+?) Will they be maintained in the future by him? If not, what is order of their importance?

he didn't show up for days so I don't think he will back soon

https://github.com/chikuzen/TMM2 since you work on TDeint

https://github.com/chikuzen/MPEG2DecPlus (this one also need HBD deblock update from https://github.com/mysteryx93/Avisynth-Deblock)

https://github.com/chikuzen/yadifmod2

https://github.com/chikuzen/DCTFilter

https://github.com/chikuzen/TEMmod

https://github.com/chikuzen/TCannyMod (I have fork for names changes https://github.com/realfinder/TCannyMod)

https://github.com/chikuzen/CombMask (also I have fork for names changes https://github.com/realfinder/CombMask)

https://github.com/chikuzen/ReduceFlicker

https://github.com/chikuzen/PlanarTools

[FranceBB]

RescueMyTime reported 300+ hours for the last three months (plus my official work), I'm so glad that I'm saving two hours per day on commute :) thanks to this stupid covid.

(Out of couriosity, do other developers watch films, videos or read books?)




First of all, thank you for all the time you spent on Avisynth for this community.
If it wasn't for you we would have been in a way worse situation right now.
If Avisynth is used across a wide range of users (both individuals and companies) today is thanks to all the developers but in particular thanks to you.
You not only kept the core, the frameserver itself, updated, but you also modernized a lot of plugins.
And... sure, you could have just been sitting on the couch watching TV all day like many people are doing during this pandemic, but thanks God you didn't; instead, you spent a lot of hours of your own spare time on Avisynth and we're really thanking you for this.
Every evening, in the UK, people are clapping for healthcare workers and their efforts; well, Ferenc, Doom9 is clapping for you.
Thank you for everything you've done.
Keep up the good work,
Frank.

[Groucho2004]

First of all, thank you for all the time you spent on Avisynth for this community.
...
If Avisynth is used across a wide range of users (both individuals and companies) today is thanks to all the developers but in particular thanks to you.
...
And... sure, you could have just been sitting on the couch watching TV all day like many people are doing during this pandemic, but thanks God you didn'tYou have to make up your mind, either thank Ferenc or your imaginary friend.

[MeteorRain]

Chikuzen is active on twitter though. But haven't seen him talking on slack or here.

https://twitter.com/AviSynthPlus/status/1260366139500007424?s=19

[zambelli]

I'm running into some strange colorspace/bitdepth conversion issues in AviSynth+ 3.5 r3106.

For example, this script works fine, returns RGB64:
ColorBars(1920,1080, "YV12")
ConvertBits(16)
ConvertToRGB64(matrix="Rec709")

This script also works fine, also returns RGB64:
ColorBars(1920,1080, "YV12")
ConvertToRGB64(matrix="Rec709", chromaresample="spline36")

However, this script throws a "ConvertToRGB: ChromePlacement and ChromeResample options are not supported" error:
ColorBars(1920,1080, "YV12")
ConvertBits(16)
ConvertToRGB64(matrix="Rec709", chromaresample="spline36")
I don't see why it wouldn't be supported though - U and V in YUV420P16 are quarter resolution, so chroma would need to be resampled to produce RGB64.

Finally, this script crashes both VirtualDub2 and FFmpeg:
ColorBars(1920,1080, "YV12")
ConvertToRGB64(matrix="Rec709")
ConvertBits(8, dither=1)

Bugs or user error?

[Reel.Deel]

So what about chikuzen's plugins (which have to be rebuilt for next Avs+?) Will they be maintained in the future by him? If not, what is order of their importance?

asd posted some updated plugins with the v8 interface on the AviSynth+ x64 plugins page on the wiki: (http://avisynth.nl/index.php/AviSynth%2B_x64_plugins)


(Added CombMask v8 interface)
(Added DCTFilter v8 interface)
(Added RawSourcePlus with fixed 10/12/14-bit input, v8 interface)
(Added ReduceFlicker v8 interface)
(Added TMM2 v8 interface)


I would be nice to have chikuzen's updated plugins on GitHub, much more reliable then random download links as they are now. Also, some of the plugins have additional commits and there was never a formal release.

[pinterf]

asd posted some updated plugins with the v8 interface on the AviSynth+ x64 plugins page on the wiki: (http://avisynth.nl/index.php/AviSynth%2B_x64_plugins)


(Added CombMask v8 interface)
(Added DCTFilter v8 interface)
(Added RawSourcePlus with fixed 10/12/14-bit input, v8 interface)
(Added ReduceFlicker v8 interface)
(Added TMM2 v8 interface)


I would be nice to have chikuzen's updated plugins on GitHub, much more reliable then random download links as they are now. Also, some of the plugins have additional commits and there was never a formal release.
I wonder if they are simply recompiled or adaptively using the frame-property preserving NewVideoFrameP as well. So are they are good only for the new v8 or can work with any former avs versions?

Yes, git is a must have. On one hand now we have to download them and make manual source-code comparison to see what has been changed.

When they are getting new features (such as 10+ bits) in the future the sources have to be put back to git. And because of the credits and source history they have to be forked from the original repository and the new developer must either forget or reinvent these "offline" changes.

I'd encourage everyone to use github in order not to break development history.

Fork, modify, commit, document, release.

[kedautinh12]

I find out a fork from maki's MPEG2DecPlus 0.1.1 on github, is it added v8 interface or not??
https://github.com/299792458m/DGIndex_mod

[tormento]

I'd encourage everyone to use github in order not to break development history.
Perhaps who maintains wiki x64 plugin page could use GitHub for "lost" authors or not willing to.

[tormento]

Developers, look at Intel Parallel C++ (https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/dpc-compiler.html).

It promises to reuse code across hardware targets (CPUs and accelerators such as GPUs and FPGAs) and also perform custom tuning for a specific accelerator.

I have access to Intel Resources from University but I am not a programmer. Perhaps someone could get a look and start to think about moving AVS+ plugins to next stage?

[MeteorRain]

There was already C++AMP. I used that a few years ago but it's still not popular till today.

[pinterf]

I find out a fork from maki's MPEG2DecPlus 0.1.1 on github, is it added v8 interface or not??
https://github.com/299792458m/DGIndex_mod

Note that v8 update will only be necessary if plugin is using the non-finalized IScriptEnvironment2 Avisynth+ interface calls.

Plugins which were not relying on IScriptEnvironment2 will be working fine.

[jpsdr]

And... sure, you could have just been sitting on the couch watching TV all day like many people are doing during this pandemic...
Yes i did... A lot of animes, some old 80's SF/Heroic Fantasy cheaped movies, and also reading manga and comics...:D

[pinterf]

My today's recommendation: The Plague by Albert Camus. I've read it in last November, pandemic started in December.

[StainlessS]

Dont blame your self P, you can in no way be held responsible for this bloody lockdown, coincidence only, really.

EDIT: to below.
David Attenborough, yep, on much safer ground there :)

[pinterf]

Dont blame your self P, you can in no way be held responsible for this bloody lockdown, coincidence only, really.
Yep, though my currently finished book is neutral, I hope (Zoo Quest by David Attenborough)

[Reel.Deel]

I wonder if they are simply recompiled or adaptively using the frame-property preserving NewVideoFrameP as well. So are they are good only for the new v8 or can work with any former avs versions?

Yes, git is a must have. On one hand now we have to download them and make manual source-code comparison to see what has been changed.

When they are getting new features (such as 10+ bits) in the future the sources have to be put back to git. And because of the credits and source history they have to be forked from the original repository and the new developer must either forget or reinvent these "offline" changes.

I'd encourage everyone to use github in order not to break development history.

Fork, modify, commit, document, release.

Ask, and you shall receive: https://github.com/Asd-g?tab=repositories

asd created a GitHub account, the modified plugins from chickuzen are there. by looking at the commits I don't think they are just recompiled, then again I am no programmer :).


******
asd: if you are reading this, I just want to say thank you for all of the plugins you've uploaded on the wiki. Please check the wiki talk page in the coming days, I have a question for you. Thanks again

[pinterf]

Ask, and you shall receive: https://github.com/Asd-g?tab=repositories

asd created a GitHub account, the modified plugins from chickuzen are there. by looking at the commits I don't think they are just recompiled, then again I am no programmer :).

Great, thank you, Asd-g, you spent quite a few hours with the updates.

[kedautinh12]

Asd-g, if you are reading this post, can port back there plugins from vapoursynth
https://forum.doom9.org/showpost.php?p=1909447&postcount=841

[tormento]

I find out a fork from maki's MPEG2DecPlus 0.1.1 on github, is it added v8 interface or not??
https://github.com/299792458m/DGIndex_mod
This (https://github.com/Asd-g/MPEG2DecPlus) for sure is, looking at sources.

[videoh]

What is this "v8 interface" you are talking about?

[pinterf]

What is this "v8 interface" you are talking about?

It's still in a development branch called neo.

IScriptEnvironment has been extended with some new functions which broke those plugins which were using avs+ specific IScriptEnvironment2. So such plugins have to be rebuilt with adaptively using new IScriptEnvironment functions.

The biggest change is the frame property support.
Just like in VS.
It is only the framework. Interface support and fp passing throughout all internal Avisynth+ plugins.

Ideally every filter in the filter chain has to able to pass frame properties on. Such plugins should use a V8 interface NewVideoFrameP instead of NewVideoFrame. But only if V8 host is detected.

Like this:

Technique is very simple.

Read about the changes:
https://github.com/AviSynth/AviSynthPlus/blob/neo/distrib/Readme/readme_history.txt

Grab new Avisynth headers (at the moment from here):
https://github.com/AviSynth/AviSynthPlus/tree/neo

Check V8 availability on create:
https://github.com/pinterf/TIVTC/blob/master/src/TDeint/TDeinterlace.cpp#L261

Use the new feature adaptively
https://github.com/pinterf/TIVTC/blob/master/src/TDeint/TDeinterlacePlanar.cpp#L169

Or when you have a source filter you can add frame properties just like in VapourSynth.

Obviously it is just a framework it will require many more steps later to use this feature properly and along with specific convertions and rules.

[Myrsloik]

Interesting and familiar looking stuff. What happens with plugins that don't pass along properties? They all just disappear or you use some simple rule to try to pass them along anyway?

[kedautinh12]

This (https://github.com/Asd-g/MPEG2DecPlus) for sure is, looking at sources.

I already seen it, thanks

[videoh]

It's still in a development branch called neo.
...
Obviously it is just a framework it will require many more steps later to use this feature properly and along with specific convertions and rules.
Thank you for the explanation. I suppose then it's not something to worry about quite yet as a filter author.

[pinterf]

Interesting and familiar looking stuff. What happens with plugins that don't pass along properties? They all just disappear or you use some simple rule to try to pass them along anyway?
Familiar, yes. Thank you in advance, I've never thought that the concept would work in any Avisynth without breaking everything.

Plugins which do not pass properties will unfortunately break the chain.

[videoh]

Is there any documentation of the properties and how to use them? Couldn't make much sense out of it from looking at avisynth.h. Thank you.

[tormento]

asd created a GitHub account, the modified plugins from chickuzen are there. by looking at the commits I don't think they are just recompiled, then again I am no programmer :).
How I wish everybody put dll version too in the compiled build.

[Myrsloik]

Familiar, yes. Thank you in advance, I've never thought that the concept would work in any Avisynth without breaking everything.

Plugins which do not pass properties will unfortunately break the chain.

You can't do something like my avisynth compatibility layer? Copy the properties from the input frame in secret?

(of course this in some cases need plugin parameter parsing and a deep knowledge of framerate modifications for plugins that do such things)

[pinterf]

You can't do something like my avisynth compatibility layer? Copy the properties from the input frame in secret?

(of course this in some cases need plugin parameter parsing and a deep knowledge of framerate modifications for plugins that do such things)
At the moment I cannot imagine how could I do that in a consistent way with parameter parsing.

Anyway when things are settling down I'd like to reproduce an issue with VapourSynth with which I had troubles and had to fix.

But probably you can answer it right now.

I added property "X" to a frame.
Then later (in another "ScriptClip") I added property "X" index #1 (2nd array element).

This array addition was not thread safe, second addition appeared multiple times in propShow when used in an MT environment. So the container was not detached in this case.

The fix was in this commit
https://github.com/AviSynth/AviSynthPlus/commit/f38951eb978def978a35ea210148e05c60117290

Probably this one has no proper use case but I encountered the issue during my mt stress tests.