RemoveDirt - pfmod

[pinterf]

RemoveDirt v1.0 (20250108)
https://github.com/pinterf/RemoveDirt/releases/tag/v1.0
Milestone: Vapoursynth (API v4) support

RemoveDirt v0.9.2 (20190324)
https://github.com/pinterf/RemoveDirt/releases/tag/v0.9.2

Link to the previous topic:
https://forum.doom9.org/showthread.php?s=&threadid=70856


# RemoveDirt - pfmod
A plugin for removing dirt from film clips.

Port of classic RemoveDirt 0.9 to Avisynth v2.6 interface (x86/x64), adding new color spaces, bugfixes.

Functions in plugin: RestoreMotionBlocks and SCSelect.
This mod does not support other filters appearing in 1.0beta.

Note: Previous v0.9 DLL versions named differently (RemoveDirtT.DLL, RemoveDirtSSE2.DLL) should be deleted from your plugin folder.

- (20190324 v0.9.2)
- RestoreMotionBlocks: 10-16 bit support. Relevant threshold and noise parameters are bit depth independent.
- minor speedup
- SCSelect: add support for 10-16 bits and 32 bit float clips
- SCSelect: add support for planar RGB
- FIX: SCSelect: make it work properly for large frames (>8MPixel)
- FIX: SCSelect: Makes use the whole frame: now counts the rightmost non-mod32 pixels as well.
- Clang 7.0.1 support (LLVM) with Visual Studio 2017 (15.9.9) (LLVM 8.0.0: not tried yet)
- Install LLVM 7.0.1 (http://releases.llvm.org/download.html, Windows pre-built libraries)
- Install Clang Power Tools & LLVM Compiler Toolchain
- https://marketplace.visualstudio.com/items?itemName=caphyon.ClangPowerTools
- https://marketplace.visualstudio.com/items?itemName=LLVMExtensions.llvm-toolchain
- update html docs
- add clang-built DLLs to the released version

- (20190314 v0.9.1)
- project moved to github: https://github.com/pinterf/RemoveDirt
- built using Visual Studio 2017
- x64 build for Avisynth+
- Added version resource to DLL
- Changed to AVS 2.6 plugin interface
- Fix: RestoreMotionBlocks: grey=false: it was copying 8x4 chroma pixels for YV12 and 8x8 for YUY2 instead of 4x4 and 8x4 blocks
- Fix: SCSelect: Old v0.9 SSE2 code omitted difference checking for every second 8 columns
- Removed MMX code, now requires SSE2
- Reports MT Modes for Avisynth+: MT_SERIALIZED for SCSelect
- Reports MT Modes for Avisynth+: MT_MULTI_INSTANCE for RestoreMotionBlocks (may not be any faster)
- Added Y, YV16 and YV24 support besides existing YV12 and planar-hacked-YUY2

- (20190312)
Initial source v0.9 (2005/05/07) moved to VS2017 project

Links
=====
Project: https://github.com/pinterf/RemoveDirt
Forum: https://forum.doom9.org/showthread.php?t=176199
Additional info: http://avisynth.nl/index.php/RemoveDirt

[pinterf]

Finally I used the original Avisynth source, the VapourSynth port had bugs, I'll point to those places on my fork later next week.

[poisondeathray]

Thanks

[SaurusX]

This is much appreciated.

[filler56789]

Thanks!

[Reel.Deel]

Thank you pinterf, I will update the wiki in the next couple of days.

[ChaosKing]

I think the VS port wasn't finished yet. The one dll I found crashes often. Thx for the release.

I always wondered how hard it is to make "hybrid plugins" now which supports vs and avs+, since avs+ now supports 16bit and the same color spaces? Some plugins like this exists: knlmeans, f3kdb, ffms2
Or is it just too cumbersome because of the different threading models?

p.s. You can also install RemoveDirt via avsrepo now

[real.finder]

been waiting for this, thanks

[Myrsloik]

I think the VS port wasn't finished yet. The one dll I found crashes often. Thx for the release.

I always wondered how hard it is to make "hybrid plugins" now which supports vs and avs+, since avs+ now supports 16bit and the same color spaces? Some plugins like this exists: knlmeans, f3kdb, ffms2
Or is it just too cumbersome because of the different threading models?

p.s. You can also install RemoveDirt via avsrepo now

It's generally very simple. The threading models are extremely similar.

[pinterf]

New build.
Download RemoveDirt v0.9.2 (https://github.com/pinterf/RemoveDirt/releases/tag/v0.9.2)
Just now: experimental (and for me quicker) clang build inside.
Original code is almost completely rewritten by now.

- (20190324 v0.9.2)
- RestoreMotionBlocks: 10-16 bit support. Relevant threshold and noise parameters are bit depth independent.
- minor speedup
- SCSelect: add support for 10-16 bits and 32 bit float clips
- SCSelect: add support for planar RGB
- FIX: SCSelect: make it work properly for large frames (>8MPixel)
- FIX: SCSelect: Makes use the whole frame: now counts the rightmost non-mod32 pixels as well.
- Clang 7.0.1 support (LLVM) with Visual Studio 2017 (15.9.9) (LLVM 8.0.0: not tried yet)
- Install LLVM 7.0.1 (http://releases.llvm.org/download.html, Windows pre-built libraries)
- Install Clang Power Tools & LLVM Compiler Toolchain
- https://marketplace.visualstudio.com/items?itemName=caphyon.ClangPowerTools
- https://marketplace.visualstudio.com/items?itemName=LLVMExtensions.llvm-toolchain
- update html docs
- add clang-built DLLs to the released version

[real.finder]

New build.
Download RemoveDirt v0.9.2 (https://github.com/pinterf/RemoveDirt/releases/tag/v0.9.2)
Just now: experimental (and for me quicker) clang build inside.
Original code is almost completely rewritten by now.


:thanks:

any plan to port RemoveGrainT to x64? it only one missed for run RemoveDirtMC_SE in x64

[pinterf]

:thanks:

any plan to port RemoveGrainT to x64? it only one missed for run RemoveDirtMC_SE in x64

Which function is used?

[real.finder]

Which function is used?

TemporalRepair in TemporalDeGrain_SE

[FranceBB]

New build.
Download RemoveDirt v0.9.2 (https://github.com/pinterf/RemoveDirt/releases/tag/v0.9.2)
Just now: experimental (and for me quicker) clang build inside.
Original code is almost completely rewritten by now.

- (20190324 v0.9.2)
- RestoreMotionBlocks: 10-16 bit support. Relevant threshold and noise parameters are bit depth independent.
- minor speedup
- SCSelect: add support for 10-16 bits and 32 bit float clips
- SCSelect: add support for planar RGB
- FIX: SCSelect: make it work properly for large frames (>8MPixel)
- FIX: SCSelect: Makes use the whole frame: now counts the rightmost non-mod32 pixels as well.
- Clang 7.0.1 support (LLVM) with Visual Studio 2017 (15.9.9) (LLVM 8.0.0: not tried yet)
- Install LLVM 7.0.1 (http://releases.llvm.org/download.html, Windows pre-built libraries)
- Install Clang Power Tools & LLVM Compiler Toolchain
- https://marketplace.visualstudio.com/items?itemName=caphyon.ClangPowerTools
- https://marketplace.visualstudio.com/items?itemName=LLVMExtensions.llvm-toolchain
- update html docs
- add clang-built DLLs to the released version

Thank you! :D
Out of curiosity, what are the benefits of the Clang compiled version over the normal x86 version?

Thank you in advance.

[pinterf]

Clang sometimes generates quicker code, it seems that it features with smarter optimizer than MS. For example the above mentioned TemporalRepair has modes that run 10-15% faster in clang version than with MS built DLL.
I was also looking at the generated assembly code based on pure C source. clang was able to vectorize (parallel computing of pixel data using mm registers) situations where MS simply stuck with linear one-by-one processing.

[tormento]

Clang sometimes generates quicker code, it seems that it features with smarter optimizer than MS.
It would be nice to try your MVTools, MaskTools and RgTools with Clang...

[real.finder]

I just tried clang in xp in vm, it's can't load the dll with error 0x36b1, normal one work fine, clang 64 bit work fine in windows server 2016

[tormento]

I just tried clang in xp in vm, it's can't load the dll with error 0x36b1, normal one work fine, clang 64 bit work fine in windows server 2016
XP is 18 years old. Time for it to go out from parents' home :p

[pinterf]

I just tried clang in xp in vm, it's can't load the dll with error 0x36b1, normal one work fine, clang 64 bit work fine in windows server 2016
LLVM (clang) dropped XP support three years ago, since version 3.8. Now I'm using latest version (8.0)

[StainlessS]

It would be nice to try your MVTools, MaskTools and RgTools with Clang...

Not if it totally breaks XP compatibility for only small speed increase.
EDIT: Removedirt v0.9.1 looks to be XP compatible.

[tormento]

Not if it totally breaks XP compatibility for only small speed increase.
Again this XP debate. And again and again.

Who cares about XP anymore? 2%? 3%?

XP can't even correctly manage modern hardware and amount of memory needed to make some video conversion.

Let's be serious.

Anyway, nobody wants to prevent Pinterf to release dual versions...

[Groucho2004]

Again this XP debate. And again and again.You were the first taking the bait. :D

XP can't even correctly manage modern hardware and amount of memory needed to make some video conversion.
You're correct about managing modern hardware.

However, XP64/WinServer2003-64 can certainly manage large amounts of memory, even more than some consumer versions of newer Windows OS's.

Let's be serious.Why? I prefer to have some fun.

[pinterf]

Not if it totally breaks XP compatibility for only small speed increase.
EDIT: Removedirt v0.9.1 looks to be XP compatible.
0.9.2 should be compatible as well, my clang builds are extras.

[tormento]

However, XP64/WinServer2003-64 can certainly manage large amounts of memory, even more than some consumer versions of newer Windows OS's.
I really can't understand this attachment to ancient tech, like if we were in a steampunk world. There are no drivers for modern video cards or chipsets.

Nobody prevents people to drive a Ford model T car but they can't keep on asking and pretending forever to have spare parts and correct octane fuel from gas stations when they are a tiny minority.

Sometimes the price (i.e. time) to supply ancient technology the support they need becomes simply too much, if not even harmful and detrimental for the majority.

[tormento]

0.9.2 should be compatible as well, my clang builds are extras.
I strongly appreciate your trials to give us faster and modern products.

[tormento]

However, XP64/WinServer2003-64 can certainly manage large amounts of memory, even more than some consumer versions of newer Windows OS's.
Not true (https://docs.microsoft.com/en-us/windows/desktop/memory/memory-limits-for-windows-releases).

[Groucho2004]

I really can't understand this attachment to ancient tech, like if we were in a steampunk world. There are no drivers for modern video cards or chipsets.

Nobody prevents people to drive a Ford model T car but they can't keep on asking and pretending forever to have spare parts and correct octane fuel from gas stations when they are a tiny minority.

Sometimes the price (i.e. time) to supply ancient technology the support they need becomes simply too much, if not even harmful and detrimental for the majority.
I simply pointed out your false statement and you keep rambling on with metaphors.
Did someone hit you with XP product boxes when you were a boy?

[tormento]

I simply pointed out your false statement
Which one? Memory? I replied you with the link.

[Groucho2004]

However, XP64/WinServer2003-64 can certainly manage large amounts of memory, even more than some consumer versions of newer Windows OS's.
Not true (https://docs.microsoft.com/en-us/windows/desktop/memory/memory-limits-for-windows-releases).
Limits:
Windows Server 2003 SP2 x64: 1 TB
Windows XP x64: 128 GB

Windows Vista Home Premium x64: 16 GB
Windows Vista Home Basic x64: 8 GB

Windows 7 Home Premium x64: 16 GB
Windows 7 Home Basic x64: 8 GB

[Groucho2004]

I simply pointed out your false statement
Which one? Memory? I replied you with the link.
I only pointed out one false statement. Maybe you read the posts to which you reply?

[tormento]

I only pointed out one false statement. Maybe you read the posts to which you reply?

Windows 10 Home x64 128 GB

Nobody would be so dumb to upgrade to a modern os and go to Vista.

Upgrade from 7 to 10 is free. And a license for 10 is 7/10€ on Amazon. Have I to say more? You defense of indefensible is boring and illogical.

We went far OT. I will stop here.

[Groucho2004]

Windows 10 Home x64 128 GB

Nobody would be so dumb to upgrade to a modern os and go to Vista.

Upgrade from 7 to 10 is free. And a license for 10 is 7/10€ on Amazon. Have I to say more? You defense of indefensible is boring and illogical.
Everything in your post is beside the point. You made a false statement, I corrected you. After that you just posted random ramblings and nonsense such as the price of Windows 10 on Amazon and that people would be dumb to use an OS that you don't like.

You argue like a 5 year old and logical thinking does not appear to be your forte so you should not accuse others of being illogical.

[StainlessS]

0.9.2 should be compatible as well, my clang builds are extras.


Oops, re-downloading ... :)

I like Windows 2000 best, whats the chance of a New version AVS+ RemoveDirt etc, supporting this ol' classic ? https://www.cosgan.de/images/midi/ekelig/e010.gif

[pinterf]

Oops, re-downloading ... :)

I like Windows 2000 best, whats the chance of a New version AVS+ RemoveDirt etc, supporting this ol' classic ? https://www.cosgan.de/images/midi/ekelig/e010.gif
Yeah, win2k. Good idea. One would dig out further archaic things from memories, pre-SSE2, XP, I propose all of you visiting The 25th Hour (see the books of Abarat) for ideas like supporting ZX Spectrum :) Ahh, it means only some extra day, perhaps weeks or years (the worst case) for programmers who are forced to maintain compatibility and do releases but it definitely worth to see the smile on the faces of the lucky ones owning a vintage hardware.

(yep, it's really a good idea to re-read Abarat - Clive Barker is a genius, I recommend you all, much better than encoding stupid series)

[hello_hello]

Upgrade from 7 to 10 is free.

How does that help XP users?

And a license for 10 is 7/10€ on Amazon. Have I to say more? You defense of indefensible is boring and illogical.

Where on Amazon? I saw a few Windows guides for that price, but the cheapest Win10 I found on Amazon.com was just under $100. My local computer shop (Australia) have it at $150 for Win10 Home and $200 for Win10 Pro, and that's for OEM versions, not retail.

[tormento]

Clive Barker is a genius
Agree.

[tormento]

My local computer shop (Australia) have it at $150 for Win10 Home and $200 for Win10 Pro, and that's for OEM versions, not retail.
If you don't want to "break" the OEM license agreement, you can find legitimate Windows 10 Pro ESD retail keys for 59€ here in Italy

Anyway, the smartest thing is to buy a Windows 7 Pro ESD for about 25€ and then upgrade for free by manually downloading Microsoft Upgrade Assistant, still available from Microsoft.

[jpsdr]

...for ideas like supporting ZX Spectrum :)...

ZX Spectrum... My youth (middle school) and my beginig in the computer world...:p

[tormento]

ZX Spectrum... My youth (middle school) and my beginig in the computer world...:p

Mine too. I suppose we are same age :o

[jpsdr]

Probably globaly the same generation. Made a mistake/confusion... ZX spectrum i was more in high school. It was the ZX81 i was middle school...

Yes pinterf, can you make a ZX81 compatible version...? (And it's the good day for this kin of things) :D

[pinterf]

Probably globaly the same generation. Made a mistake/confusion... ZX spectrum i was more in high school. It was the ZX81 i was middle school...

Yes pinterf, can you make a ZX81 compatible version...? (And it's the good day for this kin of things) :D
O.K. but it has to work with the 1 kbyte version as well :).
(I was only window-shopping when I was 13 or so. Unfortunately I possessed none of the popular machines (C64 and Speccy), they costed us too much, I had only a DIY kit based Z80 machine with 6 kbytes RAM during my 15-18 years old era. Optimization, yes, was always welcome)

[StainlessS]

Arh, my ol' ZX81 with 32KB ram pack (complete with lump of blue-tac or optional RAMPACK wobble feature).
After buildin' the ZX81 (with soldering iron), was a hairy moment when nothing happened on switch on,
was heat sink not quite square with board and slightly overlapped the board edge causing short with
earth strip around edge of board, squared up with board, all okey-dokey.
Boxed as new ZX Spectrum now bout £1500, or so I heard.
Wish I had had the power house ZX80, what a machine !

[Groucho2004]

Boxed as new ZX Spectrum now bout £1500, or so I heard.What? That's insane.

[StainlessS]

Qwik look on ebay,most expensive found was £599.00 (for the 128 KB m/c).
https://www.ebay.co.uk/i/293003953054

I think the £1,500 price tag was mentioned in some review.

EDIT: The £599.00 Spectrum above aint quite MINT, the machine might well be, but the box looks well knocked about.
£1,500.00 tag was for mint boxed.

Mind you, a year or two ago,someone put up a Samsung Galaxy Beam GT xxx (dont remember number), a projector phone,
somebody had one on e-bay for $2,500, they were only about $300 new and many others on e-bay were about $80.00.
Guess some people just put em up at rediculous price in the hope that someone stupid comes along.
EDIT: Maybe Galaxy Beam GT i8530.

EDIT: In same link above, Sinclair ZX80 and at same price £599.00.

EDIT To below: 32KB RAMPACK was the biggest possible ram upgrade, came initially with 1KB ram (my 1st rampack was 16KB).

[FranceBB]

my ol' ZX81 with 32KB ram pack

Considering that my first computer shipped with 64MB of RAM and Windows 98, I feel like I'm much younger than you, Jean-Philippe, Tormento and Ferenc are. :P

[pinterf]

Considering that my first computer shipped with 64MB of RAM and Windows 98, I feel like I'm much younger than you, Jean-Philippe, Tormento and Ferenc are. :P
Yep, you are lucky, unfortunately we'll not live out when gravitational-wave detectors would be easily embedded in an iPhone XLIII. :)

[jpsdr]

More serious question : Is there some kind of redistribuable to install to run the clang/llvm version ?
I've tried the releases.llvm link to see, but (at least right now) didn't work (time out).

[pinterf]

I built with /MT so it does not need any redistributables.

[pinterf]

TemporalRepair in TemporalDeGrain_SE

I'm near the end of porting (including to make the source human readable, re-creating plain C language versions, these works take a huge amount of time as well). Right now I put it in the RemoveDirt DLL, I woudn't create a new DLL just for the sake of this single function. Opinion?

[real.finder]

I'm near the end of porting (including to make the source human readable, re-creating plain C language versions, these works take a huge amount of time as well). Right now I put it in the RemoveDirt DLL, I woudn't create a new DLL just for the sake of this single function. Opinion?

isn't Rgtools better for it? they was part of RemoveGrain in first place

[pinterf]

isn't Rgtools better for it? they was part of RemoveGrain in first place
That was my second thought.

[Reel.Deel]

isn't Rgtools better for it? they was part of RemoveGrain in first place

I second that.

Edit:

pinterf, I started working on the wiki documentation for RemoveDirt (http://avisynth.nl/index.php/RemoveDirt) (still not done), do you know what the cache and gcache parameters do in SCSelect? They are undocumented in Kassandro's wall of text, umm I mean documentation :).

[StainlessS]

RD heres the source,


public:
SCSelect(PClip clip, PClip _scene_begin, PClip _scene_end, PClip _global_motion, double dfactor, bool _debug, bool planar, int cache, int gcache) // the constructor
: GenericVideoFilter(clip), AccessFrame(vi.width, vi.IsYUY2()), scene_begin(_scene_begin), scene_end(_scene_end), global_motion(_global_motion), dirmult(dfactor), debug(_debug), lnr(-2)
{
if( vi.IsYV12() + planar == 0 ) AVSenvironment->ThrowError("SCSelect: only YV12 and planar YUY2 clips are supported");
CompareVideoInfo(vi, scene_begin->GetVideoInfo(), "SCSelect");
CompareVideoInfo(vi, scene_end->GetVideoInfo(), "SCSelect");
CompareVideoInfo(vi, global_motion->GetVideoInfo(), "SCSelect");
hblocks = vi.width / (2*SSE_INCREMENT);
incpitch = hblocks * (-2*SSE_INCREMENT);
scene_begin->SetCacheHints(CACHE_RANGE, 0);
scene_end->SetCacheHints(CACHE_RANGE, 0);
if(gcache >= 0) global_motion->SetCacheHints(CACHE_RANGE, 0); // used nowhere else
if(cache >= 0) child->SetCacheHints(CACHE_RANGE, cache); // child = source to ScSelect (input, or DClip in below script function)
}
};

AVSValue __cdecl CreateSCSelect(AVSValue args, void* user_data, IScriptEnvironment* env) // get user args and calls constructor returning filtered clip
{
enum ARGS { CLIP, SBEGIN, SEND, GMOTION, DFACTOR, DEBUG, PLANAR, CACHE, GCACHE };
return new SCSelect(args[CLIP].AsClip(), args[SBEGIN].AsClip(), args[SEND].AsClip(), args[GMOTION].AsClip(), args[DFACTOR].AsFloat(4.0)
, args[DEBUG].AsBool(false), args[PLANAR].AsBool(false), args[CACHE].AsInt(2), args[GCACHE].AsInt(0)); // cache default 2, gcache default 0
};


No idea how that cash stuff works, I prefer plastic :) [ EDIT: Actually, many UK banknotes are now plastic, but its just not the same. ]

EDIT: So far as I remember, the global_motion clip is the clip delivered when decided that it is NOT a scene change, and may or may not be subtitled by the client/user.

EDIT:
This is almost the same as a script func [ dont think I ever got different results comparing to SCSelect ]
EDIT: SCSelect(input) and SCSelect_Like(dclip) are synonymous except that dclip can be dissimilar colorspace and dimensions to the other script clips.

# RemoveDirt's SCSelect(clip input, clip scene_begin, clip scene_end, clip global_motion, float dfactor, bool debug, bool planar)

Function SCSelect_Like(clip dclip,clip Start,clip End,clip Motion, float "dfactor",bool "debug") {
# Start, End, Motion MUST all be same, dclip can be other colorspace/size (unlike SCSelect).
dfactor=Float(Default(dfactor,4.0))
debug=Default(debug,false)
Global SCM_A=0.0 Global SCM_B=0.0 Global SCM_SC=0 Global SCM_Prev=-1
Motion.ScriptClip("""
n = current_frame
NotNext = (n!=SCM_Prev+1)
Global SCM_A=(NotNext)? RT_LumaDifference(dclip,dclip,n=n-1,n2=n) : SCM_B
Global SCM_B= RT_LumaDifference(dclip,dclip,n=n,n2=n+1)
# 0 = Start of scene, 1 = End of scene, 2 = Global motion
Global SCM_SC=(n==FrameCount-1)?1:(SCM_A>dfactor*SCM_B || n==0)?0:(SCM_B>dfactor*SCM_A)?1:2
(SCM_SC==0) ? Start : (SCM_SC==1) ? End : Last # Choose Start, End or Motion(ie Last)
(debug)?RT_Subtitle("%d ] %6.2f %6.2f SC=%d",n,SCM_A,SCM_B,SCM_SC):NOP
Global SCM_Prev=n
Return Last
""",args="dfactor,Start,End,Dclip,debug") # Needs Grunt for args
return Last
}

[pinterf]

I second that.

Edit:

pinterf, I started working on the wiki documentation for RemoveDirt (http://avisynth.nl/index.php/RemoveDirt) (still not done), do you know what the cache and gcache parameters do in SCSelect? They are undocumented in Kassandro's wall of text, umm I mean documentation :).
They were converted internally to AVS 2.5 cache hints, telling the frame caching system to prefer caching at least the given number of frames. In Avisynth+ these hints do nothing. For avs 2.6, dunno. It probably served fine tuning purposes. Now the parameters are kept for compatibility reasons in order not to break old scripts.

[Reel.Deel]

They were converted internally to AVS 2.5 cache hints, telling the frame caching system to prefer caching at least the given number of frames. In Avisynth+ these hints do nothing. For avs 2.6, dunno. It probably served fine tuning purposes. Now the parameters are kept for compatibility reasons in order not to break old scripts.

Thanks for the info, I will add it to the wiki doc. next time I have a bit of time.

[thescrapyard]

Arh, my ol' ZX81 with 32KB ram pack (complete with lump of blue-tac or optional RAMPACK wobble feature).
After buildin' the ZX81 (with soldering iron), was a hairy moment when nothing happened on switch on,
was heat sink not quite square with board and slightly overlapped the board edge causing short with
earth strip around edge of board, squared up with board, all okey-dokey.
Boxed as new ZX Spectrum now bout £1500, or so I heard.
Wish I had had the power house ZX80, what a machine !

I have a ZX80 I acquired, comes with some paperwork and a letter from Sinclair Research plus rare white ZX80 16K RAM pack. Never been tested it works and I've had over 10 years. Keep meaning to get a proper ZX80 power supply to add to it and at least try a universal power supply to see if it actually works. Not boxed

Wish I had a Jupiter ACE though, worth over £400 even without a box and nothing else. They are really rare and did know somebody who bought one when they first came out many years ago

[StainlessS]

I have a ZX80


EDIT: In same link above, Sinclair ZX80 and at same price £599.00.


You be almost a millionaire then.

Very apt name and avatar Mr Steptoe.

[F1nkster]

How much effort would it take to get a Linux build of RemoveDirt to go with AVS+ on Linux? I'd like to request it, but I respect it may be easier said than done.

[GMJCZP]

Hi, I am having trouble with my DeCelluloid (https://forum.doom9.org/showthread.php?p=1567932#post1567932) script , some greenish frames appear, here is a sample:

Sample2 (http://www.filedropper.com/sample2)

The culprit is RemoveDirt 0.9.3 and SCSelect, I had to reuse the old RemoveDirt SSE2 1.0

[StainlessS]

You using the Clang version again with no SSE 4.1 ?
If you dont have SSE4.1, SSE2 is probably best anyway, I remember there were often problems
with SSE3 and I never used it if SSE2 were available. [speed difference, negligible].

Post your script, and versions of stuff, AVS+, RemoveDirt RGtootls.

Also, whatever problem you have, post image using this.


###
# Function DeCelluloid (adapted by GMJCZP).
###

# Eliminate garbage of top and bottom image of old videos.
# Low Gfactor values => larger detection, more probability appear artifacts
# High Gfactor values => minor detection, more safe.
# Thanks a Didée for the idea.
# https://forum.doom9.org/showthread.php?p=1567932#post1567932
# Requirements: RGTools, RemoveDirt.
# Version 1.0, first attempt.

function DeCelluloid(clip source, float "Gfactor", bool "_grey" ) {
Gfactor=default(Gfactor, 5.0) # Garbage factor
_grey = default(_grey, false) # Source in Black and White
prev = BackwardClense(source, grey=_grey)
next = ForwardClense(source, grey=_grey)
h = height(source)
h_top = int(h*0.20)
h_top = (h_top % 2) == 0 ? h_top : h_top + 1
h_bottom = int(h*0.375)
h_bottom = (h_bottom % 2) == 0 ? h_bottom : h_bottom + 1
source.LetterBox(h_top,h_bottom).SCSelect(next,prev,source,dfactor=Gfactor)
}

FN = "Sample2.mp4".RT_GetFullPathName

LWLibavVideoSource(FN)
O=Last
#return info
DeCelluloid()

TOP=StackHorizontal(O,Last)
D1=ClipDelta(Last,O)
D2=ClipDelta(Last,O,true)
BOT=StackHorizontal(D1,D2)
StackVertical(Top,Bot)
return last

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}

Off to hopsital Appt, now.
EDIT: I dont see any green stuff.

[pinterf]

Please check RgTools (bug in Forward/BackwardClense 10+ bits for non SSE4.1 processors)
https://github.com/pinterf/RgTools/releases/tag/1.2

[kedautinh12]

Thanks

[GMJCZP]

Please check RgTools (bug in Forward/BackwardClense 10+ bits for non SSE4.1 processors)
https://github.com/pinterf/RgTools/releases/tag/1.2

Thank you very much, now everything works correctly.

However, I still have the doubt, why was RGTools the one that caused the problem if SCSelect when canceling it in my tests everything worked fine? That was why I turned to the old RemoveDirt which did work.

My script is:

LwLibavVideoSource ("Sample2.MP4")
DeCelluloid()

PD: I am preemptively now using RemoveDirt 0.9.3 x86 (not clang).

[GMJCZP]

You using the Clang version again with no SSE 4.1 ?
If you dont have SSE4.1, SSE2 is probably best anyway, I remember there were often problems
with SSE3 and I never used it if SSE2 were available. [speed difference, negligible].

Post your script, and versions of stuff, AVS+, RemoveDirt RGtootls.

Also, whatever problem you have, post image using this.


###
# Function DeCelluloid (adapted by GMJCZP).
###

# Eliminate garbage of top and bottom image of old videos.
# Low Gfactor values => larger detection, more probability appear artifacts
# High Gfactor values => minor detection, more safe.
# Thanks a Didée for the idea.
# https://forum.doom9.org/showthread.php?p=1567932#post1567932
# Requirements: RGTools, RemoveDirt.
# Version 1.0, first attempt.

function DeCelluloid(clip source, float "Gfactor", bool "_grey" ) {
Gfactor=default(Gfactor, 5.0) # Garbage factor
_grey = default(_grey, false) # Source in Black and White
prev = BackwardClense(source, grey=_grey)
next = ForwardClense(source, grey=_grey)
h = height(source)
h_top = int(h*0.20)
h_top = (h_top % 2) == 0 ? h_top : h_top + 1
h_bottom = int(h*0.375)
h_bottom = (h_bottom % 2) == 0 ? h_bottom : h_bottom + 1
source.LetterBox(h_top,h_bottom).SCSelect(next,prev,source,dfactor=Gfactor)
}

FN = "Sample2.mp4".RT_GetFullPathName

LWLibavVideoSource(FN)
O=Last
#return info
DeCelluloid()

TOP=StackHorizontal(O,Last)
D1=ClipDelta(Last,O)
D2=ClipDelta(Last,O,true)
BOT=StackHorizontal(D1,D2)
StackVertical(Top,Bot)
return last

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}

Off to hopsital Appt, now.
EDIT: I dont see any green stuff.

TinMan, the madness continues.

But little by little all the problems I had, and did not know because I was away from Doom9, have been solved, The Market is flourishing again thanks to your help.

Hopefully pinterf can clarify my question about RGTools and SCSelect

[pinterf]

RemoveDirt v1.1 (20250108)
https://github.com/pinterf/RemoveDirt/releases/tag/v1.1
Milestone: Vapoursynth (API v4) support

This is my first real Vapoursynth port, so it can probably be fine tuned and may contain bugs.
Nevertheless I tried to separate the internal algorithm to have as much common part as it can be, so parts I was using in Avisynth should work in Vapoursynth as well unless I messed up obvious vsapi calls.
I was not surprised that 98% of the time went into de-Avisynthing and restructuring the existing code.

As for the Avisynth part, there is no change in it except that YUY2 (ancient planar hack) support is removed.

The DLL contains Windows builds x86 and x86, Microsoft and LLVM compiled versions.

For usage check documentation in release pack.

Thanks to Selur for the never ending motivation reminders :). By seeing the number of working hours, no wonder why I postponed this conversion for years :)

EDIT: SCSelect fix in v1.1, thx to Selur for the quick test.

[Selur]

Thanks to Selur for the never ending motivation reminders :).
Thanks to you for keeping up with my nagging and looking into it. :)
:thanks:

Cu Selur

[LightArrowsEXE]

As an avid VS user, also thank you! I hope we can see more ports for both frameservers in the future!

[Jamaika]

It's hard to say if this works correctly. The show option specifies which motion blocks are being changed.
Avisynth function: Clense [function, false, 4]
Avisynth function: ForwardClense [function, false, -1]
Avisynth function: BackwardClense [function, false, -1]
Avisynth function: SCSelect [function, function, function, function, true]
Avisynth function: Repair [function, function, 16, 16]
Avisynth function: Repair [function, function, 16, 16]
Avisynth function: RestoreMotionBlocks [function, function, function, function, false, 70, 1, 2, false, 10, 12, false]
Avisynth function: RemoveGrain [function, 17, 17]

[pinterf]

Thanks for the report, but please provide a meaningful script instead of this debug output.
Also clarify the bug report a bit, what's wrong, the documentation or "show" does not work as intended?

[Jamaika]

I`m amateur. I'm looking at the wiki page. Test script. It seems that the RestoreMotionBlocks can't work without RgTools.
function RemoveDirt(clip input, bool "_grey", int "repmode")
{
_grey=default(_grey, false)
repmode=default(repmode, 16)
clmode=17
clensed=Clense(input, grey=_grey, cache=4)
sbegin = ForwardClense(input, grey=_grey, cache=-1)
send = BackwardClense(input, grey=_grey, cache=-1)
select = SCSelect(input, sbegin, send, clensed, debug=true)
alt=Repair(select, input, mode=repmode, modeU = _grey ? -1 : repmode )
restore=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode)
corrected=RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, gmthreshold=70, dist=1, dmode=2, debug=false, noise=10, noisy=12, grey=_grey)
return RemoveGrain(corrected, mode=clmode, modeU = _grey ? -1 : clmode )
}

Testing:
AviSource("input_v210.avi", audio=false, pixel_type="V210")
RemoveDirt(false, 16)

If debug=true, then SCSelect sends output of the following type to the DebugView utility.
Hmm,... If I only have ffmpeg then this function is unnecessary. I don't have any information displayed.

The boolean variable debug and show are used for debugging. If show=true, then the blocks, which are marked as motion blocks in the first phase are colored red, those found in the second phase are colored green and finally the motion blocks marked by postprocessing are colored blue. In this way, one can easily check whether the above variables were selected appropriately. if debug=true, then RestoreMotionBlocks sends output of the following kind to the debugview utility:
I only have one color red and blue. Which parameter increases, decreases the motion block area.
When I use RemoveDirt(true, 16) the video won't play and everything is pink.

[pinterf]

When I use RemoveDirt(true, 16) the video won't play and everything is pink.
You intentionally passed grey=true to the script function which passes it over to its various filters. I suppose in this case filters simply do not process and ignore the chroma planes thus they contain garbage. In your case you see pink colors, I can see diagonal lines.

[Jamaika]

You intentionally passed grey=true to the script function which passes it over to its various filters. I suppose in this case filters simply do not process and ignore the chroma planes thus they contain garbage. In your case you see pink colors, I can see diagonal lines.
Maybe but video doesn't play.
For amateurs ffmpeg with new RestoreMotionBlocks.
https://www.sendspace.com/file/8yufh8

[FranceBB]

RemoveDirt v1.1 (20250108)
https://github.com/pinterf/RemoveDirt/releases/tag/v1.1



Thank you! :D
And as always I'm very happy to report that... it works on XP! :)

[pinterf]

Thank you! :D
And as always I'm very happy to report that... it works on XP! :)
You're welcome! Feel free to send me a text anytime if something doesn't work for you. :)