Dogway's Filters Packs

[kedautinh12]

Originally Posted by poisondeathray

GTX 650 is too old

https://github.com/WolframRhodium/VapourSynth-BM3DCUDA

Test 8 now support compute capability 3.5 and higher

Originally Posted by poisondeathray

ahh I misunderstood;

Maybe CPU fallback ? What were the CPU's between the 2 systems?

How are you measuring OpenCL usage ? What encoder are you using ? (is it using OpenCL as well) ?

Well, my post was asking how can this be working ??

It's an i7 3930K. which only supports AVX, so BM3DCPU doesn't work, and when I only had the AMD R9 290 installed, it could not use BM3DCUDA, but now that I've installed the "old" GTX 650, the encoding app seems to be able to use both GPU's when using BM3DCUDA.

The encoding app is RipBot264, which is sadly way out of date, but still works quite well.

I have a Ryzen 3950X with an AMD RX 480 (which doesn't support CUDA), so I put an old GTX 760, and the same applies, it just works

I also have a couple of Xeon E5 2690's, with old AMD HD6850's & GTX 650's, and it works the same way

Originally Posted by kedautinh12

Test 8 now support compute capability 3.5 and higher

Hi ked, I'm not using Vapoursynth, tho...

[Boulder]

Originally Posted by Dogway

Oh, that one was also reported back in October, still no word from pinterf.

So you are saying that my QTGMC mod without propCopy is fine? Or do you want to keep debugging it?

Your debug version worked fine once I commented out line 964 which had the propCopy call.

[WolframRhodium]

BM3DCUDA may run on GTX650, but it is not guaranteed by CUDA. It should not be able to run on R9 290, the usage data may be erroneous.

[kedautinh12]

Originally Posted by TDS

Hi ked, I'm not using Vapoursynth, tho...

I say about avs ver

Originally Posted by WolframRhodium

BM3DCUDA may run on GTX650, but it is not guaranteed by CUDA. It should not be able to run on R9 290, the usage data may be erroneous.

Yes, that is why I asked the question in the first place, I was shocked when it appeared that RipBot264 WAS using the OpenCL (with BM3DCUDA).

With the Distributed Encoding feature of RipBot264 you have the option to use a specific GPU to do the encoding.

So when using BM3DCUDA, and have the R9 290 on one server/port, and the GTX 650 on another server/port?, they both appear to be encoding, and at different speeds, due the difference in the Compute Units of both cards.

However without the GTX 650, BM3DCUDA cannot be used.

Originally Posted by kedautinh12

I say about avs ver

It's just that that link sent me to the Vapoursynth version info. AFAIK.

[kedautinh12]

But name ver i said related avs
https://github.com/WolframRhodium/Va...ment-886173346

Originally Posted by kedautinh12

But name ver i said related avs
https://github.com/WolframRhodium/Va...ment-886173346

Sorry....but this link was different, but made sense.

[pinterf]

Originally Posted by Dogway

Oh, that one was also reported back in October, still no word from pinterf.

So you are saying that my QTGMC mod without propCopy is fine? Or do you want to keep debugging it?

I think that was a completely different problem (speed). To address your problem propCopy was changed then in order to not copy all the frame content but to use "SubFrame" internally (you as a script writer do not need to know what env->SubFrame is), which is able to keep frame content without actually copying any byte in its planes. The same "subframe" technique is used to make quick SeparateFields, aligned crop, ExtractX, etc.

I suppose this memory leak appeared after this change, because of this subframe-hack which I have to investigate why it doesn't release its memory or clip reference of I don't know what.

[pinterf]

Originally Posted by Dogway

I'm pretty sure it's a memory leak in pixel addressing, it has been reported almost 2 months ago, and recalled 2 weeks ago but pinterf has been turning a blind eye on it.

I will try to cobble up a version without pixel addressing so you can confirm. It's either that or a LUT bug.

Well, probably I have blind eyes but if you'd follow things on git you would realize that I'm not just sitting at home, eating pizza and watching clouds on the sky, plus I've got an official job. In last November I went to a cinema and watched Dune, that was the last time I had time watching _any_ films.

[pinterf]

Originally Posted by Boulder

I think I found the culprit after many rounds of debugging - it's propCopy and that's why your QTGMC leaks memory since STGMC doesn't use the props feature at all.

This won't stall:

DGSource("source.dgi")
STGMC(preset="medium", noisepreset="fast", tr2=0, sigma=0.25)
propSet("_FieldBased", 0)
Spline36Resize(1280,720)
SRestore()

While this will:

DGSource("source.dgi")
org=last
STGMC(preset="medium", noisepreset="fast", tr2=0, sigma=0.25)
propCopy(last, org)
propSet("_FieldBased", 0)
Spline36Resize(1280,720)
SRestore()

I think I need to report this in GitHub along with the other property related issue I found earlier.

Has someone accidentally minimize the script? I omitted SRestore and resize, replaced STGMC (this must be a new thing I'm not aware of) with QTGMC and it is still consuming more and more memory, I wonder which part of it must be kept in order to still reproduce the problem?

[kedautinh12]

Originally Posted by pinterf

Well, probably I have blind eyes but if you'd follow things on git you would realize that I'm not just sitting at home, eating pizza and watching clouds on the sky, plus I've got an official job. In last November I went to a cinema and watched Dune, that was the last time I had time watching _any_ films.

Dune is very good film for me

[pinterf]

Not only propCopy(last, org) fails but other two-clip filters as well, replacing propCopy with Stackvertical(last,org) or Merge(last,org) yields the same effect.
I have to remove functions from QTGMC one-by-one until the phenomenon stops occuring, since probably some other function call combination inside QTGMC triggers this leak.

btw:
propCopy now gets/sets everything.
Would it be good have a new parameter to be able to pass an array of property names?
Then only selected properties would only be extracted and copied?

[Dogway]

Originally Posted by Boulder

Your debug version worked fine once I commented out line 964 which had the propCopy call.

Could you test vanilla without propCopy? So we can discard pixel addressing.

Originally Posted by pinterf

I think that was a completely different problem (speed). To address your problem propCopy was changed then in order to not copy all the frame content but to use "SubFrame" internally (you as a script writer do not need to know what env->SubFrame is), which is able to keep frame content without actually copying any byte in its planes. The same "subframe" technique is used to make quick SeparateFields, aligned crop, ExtractX, etc.

Speed issues typically uncover under the hood issues. But I mean you as a mere programmer don't need to know what hysteresis is, nor even maths at all.

Originally Posted by pinterf

Well, probably I have blind eyes but if you'd follow things on git you would realize that I'm not just sitting at home, eating pizza and watching clouds on the sky, plus I've got an official job. In last November I went to a cinema and watched Dune, that was the last time I had time watching _any_ films.

Now we are two (no time even for Dune ). I do follow things on git, I was just replying to this post. You made such a fuss about not reporting but I actually do and most of it pass ignored, so I'm not going to annoy anyone that I know is so busy with other tasks as I see on git.

propCopy now gets/sets everything.
Would it be good have a new parameter to be able to pass an array of property names?
Then only selected properties would only be extracted and copied?

Thanks, finally it gets implemented. Typically it's only "_ColorRange" what I need to restore, but having also ALL as an option is useful.

QTGMC is a good Expr() testbench because it takes it to its limits. I hope Boulder can answer if vanilla without propCopy() is fine.

[Boulder]

Originally Posted by Dogway

Could you test vanilla without propCopy? So we can discard pixel addressing.

Yes, it works. The issue appears to be a particularly nasty one and reproducable with a very simple script.

https://github.com/AviSynth/AviSynthPlus/issues/270

I'd like to add my 2c to the GitHub reporting issue as an IT guy. I definitely want to bring up things but even if I do, I don't expect them to be handled. It's always up to the maintainer to decide that, I see reporting just a perfect way to keep things in mind. The issues are very easily lost in the long threads. In your case I've made a difference since you frequent the forum a lot these days.

[Dogway]

Originally Posted by Boulder

I'd like to add my 2c to the GitHub reporting issue as an IT guy. I definitely want to bring up things but even if I do, I don't expect them to be handled. It's always up to the maintainer to decide that, I see reporting just a perfect way to keep things in mind. The issues are very easily lost in the long threads. In your case I've made a difference since you frequent the forum a lot these days.

Thanks for the report. It's a matter of time, if you report to GitHub you are supposed to dedicate enough time to debug, time I don't have, and probably pinterf either.
These days I would rather finish the last touches of my scripts because I'm burned out and switch to other things.

[Boulder]

Originally Posted by Dogway

Thanks for the report. It's a matter of time, if you report to GitHub you are supposed to dedicate enough time to debug, time I don't have, and probably pinterf either.
These days I would rather finish the last touches of my scripts because I'm burned out and switch to other things.

I always try to give enough information to be useful, that's also because of my background. At work, I get quite a lot of requests for assistance which lack a lot of information so I have to either guess or ask too many questions to do something about things.

What comes to being burnt out - your scripts already work very well. It's very much possible to take a break and come back to the ideas in the summer or whenever. Of course, if there's some dependency type of bugs found, it's a bit different but I guess the most used functions are in good shape.

[Dogway]

Yes, normally my reports whenever possible are very low level examples with Expr(), but beyond that I don't have time to debug with other conditions, compare old versions, etc.

I have an internal todo list and it's pretty much over. I had Harris corner there for a few months now, and now for ExTools the only thing left is some bugs in ex_makediff() when show=1, and update propCopy() calls to latest (unreleased) AVS+ version (also to other scripts). I will probably promote it to v8.0. It's not perfect but it's where I reach for now.

The only other thing left is MasksPack, I probably can optimize ex_limitdif() further. And finish Soothe() refactor. That's all, besides the Zopti thingy and SceneStats() which is a big but fun thing to do. There are more but for the future I guess.