Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 31st December 2015, 17:37   #1  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
Dotkill - spatial dotcrawl remover

This is a new year's eve bonus from my small collection of unfinished things.

It's a filter doodle that's been sitting around for a long time without making more progress. It's a pure spatial dotcrawl remover that works after field matching or on progressive input. It generally doesn't artifact much and when it does it's rarely worse than the original dotcrawl anyway.

Arguments:
Code:
core.dotkill.DotKill(clip[, napply=1, ignorematch=False])
Usage:
Code:
clip = core.vivtc.VFM(clip, order=1)
clip = core.dotkill.DotKill(clip, napply=1)
clip = core.vivtc.VDecimate(clip)

This filter should be placed AFTER field matching with VFM if field matching is needed. The napply setting is how many times to apply the filter, values between 1 and 4 are generally sane. The higher you go the bigger chance of artifacting. The ignorematch argument is only for testing purposes and should probably never be touched.

It will most likely perform very poorly on purely interlaced content.

The plan is still to add temporal filtering some day but don't expect it anytime soon.

And don't forget to support the projects you like. Especially the ones where you don't have to donate before you can try it.

Examples:

__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet

Last edited by Myrsloik; 20th February 2017 at 10:53.
Myrsloik is offline   Reply With Quote
Old 20th February 2017, 12:43   #2  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Thank you! Can you share macOS build ?
KingLir is offline   Reply With Quote
Old 20th February 2017, 12:44   #3  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
Quote:
Originally Posted by KingLir View Post
Thank you! Can you share macOS build ?
No, it's currently a prototype filter where I try different ideas and I don't plan on releasing the source until I have added more parts to it.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 24th November 2017, 10:41   #4  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Quote:
Originally Posted by Myrsloik View Post
No, it's currently a prototype filter where I try different ideas and I don't plan on releasing the source until I have added more parts to it.
Download link is down.
BTW- any progress on this ? If not, maybe you can consider releasing the source or and/or macOS build ?
KingLir is offline   Reply With Quote
Old 24th November 2017, 10:45   #5  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
Quote:
Originally Posted by KingLir View Post
Download link is down.
BTW- any progress on this ? If not, maybe you can consider releasing the source or and/or macOS build ?
No real progress, I try some ideas now and then but most fail horribly or require more time to prototype than I can spare. I guess I could release the source since there are about 0 other spatial only dotcrawl removers and the idea is relatively simple. Will clean it up in a few days.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 24th November 2017, 11:03   #6  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Quote:
Originally Posted by Myrsloik View Post
No real progress, I try some ideas now and then but most fail horribly or require more time to prototype than I can spare. I guess I could release the source since there are about 0 other spatial only dotcrawl removers and the idea is relatively simple. Will clean it up in a few days.
Thanks !!
KingLir is offline   Reply With Quote
Old 24th November 2017, 13:53   #7  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
BTW- did you saw this script ?
https://forum.doom9.org/showthread.p...72#post1673472
KingLir is offline   Reply With Quote
Old 24th November 2017, 14:05   #8  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
Quote:
Originally Posted by KingLir View Post
Maybe I've seen it but scripts like that are just too much effort to reverse engineer.

There are some interesting tricks you could use for telecined material though (since you know fields are duplicated in a pattern) but it doesn't appear like that script uses it.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 24th November 2017, 20:54   #9  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Quote:
Originally Posted by Myrsloik View Post
Maybe I've seen it but scripts like that are just too much effort to reverse engineer.

There are some interesting tricks you could use for telecined material though (since you know fields are duplicated in a pattern) but it doesn't appear like that script uses it.
I see.

Will the source of DeDot will help ?

http://avisynth.nl/index.php/DeDot
KingLir is offline   Reply With Quote
Old 24th November 2017, 21:03   #10  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
Quote:
Originally Posted by KingLir View Post
I see.

Will the source of DeDot will help ?

http://avisynth.nl/index.php/DeDot
Linking random stuff doesn't help. I've looked at most simple ideas already
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet

Last edited by Myrsloik; 24th November 2017 at 21:43.
Myrsloik is offline   Reply With Quote
Old 24th November 2017, 22:04   #11  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Sorry if it's look like I am linking to random stuff. I only linked to it because it says that it's a Spatial dot crawl removal.
Good to know you checked and invested much in this. Much appreciated.
KingLir is offline   Reply With Quote
Old 24th November 2017, 22:47   #12  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
https://github.com/myrsloik/DotKill
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 25th November 2017, 00:08   #13  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Quote:
Originally Posted by Myrsloik View Post
Many Thanks !!

1.
I was managed to build on macOS with the following:
gcc -c -fPIC ${CFLAGS} ${CPPFLAGS} -I. $(pkg-config --cflags vapoursynth) -o main.o dotkill1.cpp
g++ -dynamiclib -undefined suppress -flat_namespace *.o -o libdotkill.dylib

2. It does seems to remove nice amount of DotCrawl but not for frames before and after scene change. It's like it takes time for it to "kick in".
Got an idea how to fix this ?

3. When you have to use de-interlacer like QTGMC ( for PAL DVD- 24fps master that was converted to 25fps ) it seems it work best when you apply DotKill before QTGMC. Does it make sense ?

Last edited by KingLir; 25th November 2017 at 00:17.
KingLir is offline   Reply With Quote
Old 25th November 2017, 00:13   #14  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
2. It's not temporal so I'd need a sample and script to figure out why that happens

3. It's the worst on interlaced since it will blur between fields. Progressive and after ivtc are the optimal cases. Before qtgmc is probably the best best place.

What is the source material anyway? Field duplicated 24fps to make it 25fps?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 25th November 2017, 00:24   #15  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Quote:
Originally Posted by Myrsloik View Post
2. It's not temporal so I'd need a sample and script to figure out why that happens

3. It's the worst on interlaced since it will blur between fields. Progressive and after ivtc are the optimal cases. Before qtgmc is probably the best best place.

What is the source material anyway? Field duplicated 24fps to make it 25fps?
Here is my test sample:
https://mega.nz/#!xwhSyCZR!pwCAjPq2q...6Sc8Sbpf056aM8

I am trying to remove dot crawl from the carpet and above the door (first scene) and from the kitchen wall white lines (second scene).
I only manage to remove some from the white lines but not all - and only with high napply values (I even tried napply=45 with the same result as 4 or 5).

Cool to know that it's best place before qtgmc - which is a must (unfortunately) in this case, following by srestore.
KingLir is offline   Reply With Quote
Old 25th November 2017, 18:28   #16  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
The scenechanges are easily explained. The source was probably smart bobbed to make it progressive before converting it to pal framerate. And it was a really shitty filter as is customary in PAL dvds. So even static detail is discarded near scene changes for no real reason and dotcrawl detection doesn't work when fields are interpolated.

The dotcrawl most likely existed in the NTSC source that was used as input for the PAL conversion. So then you destroy the original dotcrawl structure and also any possibility of recovering detail. Blend two adjacent frames together and the dotcrawl is still there.

I guess what I'm trying to say is that your source material is shit and death is the only way out.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 26th November 2017, 11:35   #17  |  Link
KingLir
Registered User
 
Join Date: Jan 2017
Posts: 143
Quote:
Originally Posted by Myrsloik View Post
I guess what I'm trying to say is that your source material is shit and death is the only way out.
Yeah, I figured

Another member helped me to remove all dots in a very complex way while preserving most of the details.
If you like, I can share the details (script lines) over PM.
KingLir is offline   Reply With Quote
Old 28th December 2017, 20:33   #18  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,035
Fixed a reference leak in it. For those of you who compile it yourselves...
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:38.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.