Avisynth+

[pinterf]

Originally Posted by tormento

mmm... video decoder errors with DGNV.

Can somebody please upload the previous version?

EDIT: confirmed... with r2043 (the only other in mirror) it works. Please upload r2069 back

Refine that error report, please. Missing ConvertToPokemon() or what?

[tormento]

Originally Posted by pinterf

Refine that error report, please. Missing ConvertToPokemon() or what?

CUDA Error!

And then Unable to create Video Decoder.

P.S: As an Ingress agent, that is a bad word to me

[Groucho2004]

Originally Posted by tormento

CUDA Error!

And then Unable to create Video Decoder.

P.S: As an Ingress agent, that is a bad word to me

How about posting the script? You should know the drill.

[tormento]

Originally Posted by Groucho2004

How about posting the script? You should know the drill.

Nothing changed, only AviSynth+ from r2069 to latest.

Anyhow:

PHP Code:

SetMemoryMax(8000)
SetFilterMTMode("DEFAULT_MT_MODE", 2)
SetFilterMTMode("ChangeFPS", 3)

SetFilterMTMode("DGSource", 3)
LoadPlugin("D:\eseguibili\media\DGDecNV\x64\DGDecodeNV.dll")
DGSource("E:\in\1_52 quinta onda, La\quinta.dgi")

ChangeFPS(last,last,true)

SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6,TV_range=true)

Prefetch(8) 


Please, anybody, upload r2069 somewhere so I can test it back.

[Groucho2004]

Originally Posted by tormento

SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6,TV_range=true)

"SMDegrain does not have a named argument "TV_range""
So, which of the 357 versions of SMDegrain are you using?

Took out the "tv_range" parameter. With "Prefetch(2)" I'm already at 750 MB graphics memory. However, it runs.

[tormento]

Originally Posted by Groucho2004

"SMDegrain does not have a named argument "TV_range""
So, which of the 357 versions of SMDegrain are you using?

Remove that parameter. Anyhow I am using this

Nobody has a r2069 copy on HD?

[Groucho2004]

Originally Posted by tormento

Remove that parameter.

I edited my post. Your script works fine for me (XP64) but I have to lower the number of threads because of memory limits.

[ultim]

Originally Posted by tormento

Remove that parameter. Anyhow I am using this

Nobody has a r2069 copy on HD?

Please wait a few hours, will be fixed today. Pinky promise. (I'd rather give you a fixed build instead of having you test an older older version with other known flaws.)

[tormento]

Originally Posted by ultim

Please wait a few hours, will be fixed today. Pinky promise. (I'd rather give you a fixed build instead of having you test an older older version with other known flaws.)

Have to encode a movie for tonight. r2069 is more than enough for me now

[ultim]

Originally Posted by tormento

mmm... video decoder errors with DGNV.

Can somebody please upload the previous version?

EDIT: confirmed... with r2043 (the only other in mirror) it works. Please upload r2069 back

remove SetFilterMTMode("DGSource", 3) until fix.

[tormento]

Originally Posted by ultim

remove SetFilterMTMode("DGSource", 3) until fix.

Wow. That was fast and easy

[MysteryX]

I did some benchmark tests of various MT implementations here.

Let's try again with r2076

Plain script

ColorBarsHD(width=800, height=600, pixel_type="YV24").killaudio().assumefps(25, 1)
SuperResXBR(Engines=1)
Prefetch(8)

MT_SERIALIZED
FPS (min | max | average):      2.450 | 1000000 | 26.85
Memory usage (phys | virt):     356 | 382 MiB
Thread count:                   68
CPU usage (average):            4%
**it is instead using MT_MULTI_INSTANCE. If I set Force=true, it freezes on init.

MT_NICE_FILTER
FPS (min | max | average):      2.522 | 1000000 | 23.71
Memory usage (phys | virt):     245 | 264 MiB
Thread count:                   28
CPU usage (average):            4%
**Was 23.51, it is slightly faster

2 Engines
FPS (min | max | average):      2.926 | 1000000 | 26.90
Memory usage (phys | virt):     289 | 309 MiB
Thread count:                   33
CPU usage (average):            4%
**Same, but higher memory usage (309MB vs 279MB)

Let's try on my full script to upscale 288p to 768p. Script includes KNLMeans, 2x SuperResXbr, InterFrame and various 16-bit tools.

MT_SERIALIZED
Again, freezes if I force MT_SERIALIZED and it used MT_MULTI_INSTANCE instead.

MT_NICE_FILTER
FPS (min | max | average):      1.414 | 1000000 | 12.88
Memory usage (phys | virt):     966 | 1058 MiB
Thread count:                   125
CPU usage (average):            29%
**Lower performance and higher memory usage. Before:
FPS (min | max | average):      3.436 | 1000000 | 13.27
Memory usage (phys | virt):     680 | 736 MiB
Thread count:                   52
CPU usage (average):            27%

No point in setting 2 engines because there are already 2 shader functions

or let's try heavier GPU processing with complex CPU work

ColorBarsHD(width=1940, height=1080).killaudio().assumefps(25, 1).ConvertToYV12()
SMDegrain(thsad=200, prefilter=2, lsb=true)
SuperResXBR(5, 1, .15, fWidth=2600, fHeight=1500, fKernel="SSim", Engines=1)
Prefetch(8)

MT_SERIALIZED
** Cannot yet test

MT_NICE_FILTER
FPS (min | max | average):      0.347 | 1000000 | 3.176
Memory usage (phys | virt):     1125 | 1162 MiB
Thread count:                   34
CPU usage (average):            29%
**Slightly higher performance with lower CPU usage

2 Engines
FPS (min | max | average):      0.233 | 1000000 | 2.715
Memory usage (phys | virt):     1197 | 1236 MiB
Thread count:                   39
CPU usage (average):            31%
**Still the same lower performance anomaly

I want to test MT_SERIALIZED but it's not yet working for me.

The higher memory usage might be due to some filters running in MT_MULTI_INSTANCE instead of MT_SERIALIZED.

[ultim]

Originally Posted by MysteryX

I did some benchmark tests of various MT implementations here.
...

I'm home now finally, and am looking at the issue tormento reported in greater detail. I'll have a fix soon as it is not hard but the consequences of the problem were pretty severe. Please redo your tests with the build I'll publish soon.

[ultim]

Okay, r2082 has an important fix for a bug that made some filters use the default MT mode instead of the specified one under certain conditions. It was the cause for tormento's problem. Additionally another case of false positives was cured for the generated diagnostic logs, and pinterf's latest work is integrated adding support for high bit-depth blur and sharpen.

[tormento]

Originally Posted by ultim

Okay, r2082 has an important fix for a bug that made some filters use the default MT mode instead of the specified one under certain conditions. It was the cause for tormento's problem. Additionally another case of false positives was cured for the generated diagnostic logs, and pinterf's latest work is integrated adding support for high bit-depth blur and sharpen.

Problem solved

[ultim]

MysteryX, I took a quick look at your source of AviSynthShader to verify something. Whenever you call GetFrame() to get your input frames, pull those calls out of your mutex lock! The way your filter operates right now is forcing the other filters it calls into mode 3! Not nice.

The occurance I linked to might not be the only one, it was just the first I've found.

[MysteryX]

Originally Posted by ultim

MysteryX, I took a quick look at your source of AviSynthShader to verify something. Whenever you call GetFrame() to get your input frames, pull those calls out of your mutex lock! The way your filter operates right now is forcing the other filters it calls into mode 3! Not nice.

The occurance I linked to might not be the only one, it was just the first I've found.

Oh! That is one is very easy to fix: move the unique_lock down a few lines.

Good catch, I never thought about that.

By the way, does the new 16-bit code have ASM optimization or is only C++ code?

[MysteryX]

Let's try the benchmarks again with r2082

Original results here

First, this line doesn't work. SetLogParams("stderr", INFO). INFO or DEBUG is not recognized.

MT_SERIALIZED still freezes with the way I have GetFrame inside the lock. If I fix that in my code, then it works.

Plain script

ColorBarsHD(width=800, height=600, pixel_type="YV24").killaudio().assumefps(25, 1)
SuperResXBR(Engines=1)
Prefetch(8)

MT_SERIALIZED
FPS (min | max | average):      1.561 | 1000000 | 23.02 (was 22.75)
Memory usage (phys | virt):     257 | 277 MiB (was 215MB)
Thread count:                   28
CPU usage (average):            2%

MT_NICE_FILTER
FPS (min | max | average):      2.510 | 1000000 | 23.70 (was 23.51)
Memory usage (phys | virt):     250 | 269 MiB (was 238mb)
Thread count:                   28
CPU usage (average):            4%

2 Engines
FPS (min | max | average):      2.169 | 1000000 | 26.80
Memory usage (phys | virt):     281 | 300 MiB
Thread count:                   33
CPU usage (average):            4%

Let's try on my full script to upscale 288p to 768p. Script includes KNLMeans, 2x SuperResXbr, InterFrame and various 16-bit tools.

MT_SERIALIZED
FPS (min | max | average):      1.778 | 1000000 | 13.03 (was 7.930)
Memory usage (phys | virt):     644 | 698 MiB (same as before)
Thread count:                   52
CPU usage (average):            27%

MT_NICE_FILTER
FPS (min | max | average):      2.977 | 1000000 | 13.25
Memory usage (phys | virt):     679 | 734 MiB
Thread count:                   53
CPU usage (average):            27%

No point in setting 2 engines because there are already 2 shader functions

or let's try heavier GPU processing with complex CPU work

ColorBarsHD(width=1940, height=1080).killaudio().assumefps(25, 1).ConvertToYV12()
SMDegrain(thsad=200, prefilter=2, lsb=true)
SuperResXBR(5, 1, .15, fWidth=2600, fHeight=1500, fKernel="SSim", Engines=1)
Prefetch(8)

I keep getting this warning
WARNING: Caches have been shrunk due to low memory limit. This will probably degrade performance. You can try increasing the limit using SetMemoryMax().

MT_SERIALIZED
FPS (min | max | average):      0.247 | 1000000 | 3.002 (was 1.729)
Memory usage (phys | virt):     1091 | 1121 MiB (was 1044mb)
Thread count:                   34
CPU usage (average):            27%

MT_NICE_FILTER
FPS (cur | min | max | avg):    3.368 | 0.349 | 1000000 | 3.108
Memory usage (phys | virt):     1084 | 1117 MiB
Thread count:                   34
CPU usage (current | average):  27% | 28%

2 Engines
FPS (min | max | average):      0.149 | 1000000 | 2.536 (was 2.713)
Memory usage (phys | virt):     1224 | 1261 MiB
Thread count:                   39
CPU usage (average):            31%
** this last one has gone worse than before, I'm wondering what's going on here

MT_SERIALIZED is now working much better than before, and MT_NICE_FILTER with a lock still works better.

The under-utilization of the GPU when running a single engine has been improved, somehow.

[pinterf]

Could you benchmark the last one with
a.) higher SetMemoryMax(), e.g. 2500? In such low memory situation the continuous cache shrink can really degrade performance
b.) Prefetch sequence (4-5-6-7), maybe the sweet spot is not at 8, even if you feed the script into avsmeter instead of running an encoder after it.

New high bit-depth filter routines are mostly in c. The main goal now is to have a working version of them, later they can get simd optimalization.

[ultim]

Thanks for the tests. Nice to see the improvements to MT_SERIALIZED are really paying off, in your case they practically doubled the performance.

Originally Posted by MysteryX

First, this line doesn't work. SetLogParams("stderr", INFO). INFO or DEBUG is not recognized.

Yep it doesn't, because you're using it wrong. As documented in the announcement post, the constants are called LOG_INFO, LOG_DEBUG etc.

Originally Posted by MysteryX

I keep getting this warning
WARNING: Caches have been shrunk due to low memory limit. This will probably degrade performance. You can try increasing the limit using SetMemoryMax().

This is one of the many new warnings introduced in r2069, helping you diagnose performance and other problems. Having to call SetMemoryMax() in AviSynth to increase the available memory is nothing new and goes way back to before Avs+ even existed. What is new now is that now you get notified if using it would be beneficial.