Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
17th July 2012, 08:47 | #1 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Dot Crawl issue
Hello,
I know this subject has been talked over many times...howhever, in spite of the numerous threads dealing with Dot Crawl, I still can't manage to get rid of it. So let's put the record straight : I am currently working at a video compression firm and I am trying to correct artefacts like that dotcrawl on many frames. Here is the problematic one : First of all, do you agree it is indeed dot crawl ? Then my problem is I can't correct it without giving rise to harmful secondary effects typically blending on the hair or edges. Of course I still haven't been able to completely eliminate dotcrawl from that frame above. I've tried this : DeDot(luma2d=255, lumaT=25, chromaT1=0, chromaT2=255) LUTDeCrawl(ythresh=10, cthresh=10, maxdiff=50, scnchg=25, usemaxdiff=true, mask=false) It is not worth I show you the bad results I get, either I won't show you the Checkmate() one in so far it blurs too much. I've tried to ajust them with no sucess. Then how the hell could I correct that frame without reverberations where not intended ? Only the face and the borders should be processed ! |
17th July 2012, 10:43 | #2 | Link |
Registered User
Join Date: Mar 2009
Location: Germany
Posts: 5,769
|
Dot crawl can 100% sure be diagnosticised only on moving images
Dot crawl is caused by chroma interference with luma. So it should appear on the neck too as the same conditions apply there too.
__________________
Born in the USB (not USA) |
17th July 2012, 10:54 | #3 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Thanks, here is another examples :
(For the second one, I ignore rainbow) Assuming it is dot crawl, what would you to correct it in accordance with my previous expectations ? Last edited by sirt; 17th July 2012 at 11:12. |
17th July 2012, 11:09 | #4 | Link |
Big Bit Savings Now !
Join Date: Feb 2007
Location: close to the wall
Posts: 1,531
|
Yes, that looks like dot crawl. The saturation borders tell.
Rainbowing is in there as well. The left/right railing shows it. I had good success with FFT in cases of dot crawl from SVHS recordings. First you want to check which planes Y or (UV) are affected. For Y F2Quiver with pairs of elliptical filters (nef=2,...)may help. For UV F2QuiverUV with gp4 or bp4. First test=true, then find the frequency spots and apply. Both FFT filters eat up their domain, so need refixing. F2Quiver eats up Y, F2QuiverUV eats up saturation and shifts hue. You got to find out.
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain) "Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..." Last edited by Emulgator; 17th July 2012 at 11:11. |
17th July 2012, 11:21 | #6 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Emulgator
Do you refer to FFT3Dfilter ? I have never went in for which planes are affected, so that's new to me ! Assuming I find the good ones, do you mean FFT3Dfilter can do the trick ? jmac698 Thanks for your interest. I would be in rapture you to give me some ideas, unfortunately I can't share a short clip in so far - as I said - I am currently doing postprocessing at a video compression firm and the source is a YUV Uncompressed one that belongs to them. |
17th July 2012, 11:23 | #7 | Link |
Big Bit Savings Now !
Join Date: Feb 2007
Location: close to the wall
Posts: 1,531
|
Not FFT3D, but VCMohan's F2Quiver.
http://avisynth.org/vcmohan/ In my case the F2Quiver/UV combo left all details untouched and only removed what I intended to. But setting the filters and their appropriate compensation may well take hours to adjust. I would have tried Fizick's Defreq as well, but I haven't gotten it to work. The frequency pattern showed, but I saw no effect when applied. Maybe driver error on my side...
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain) "Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..." Last edited by Emulgator; 17th July 2012 at 11:31. |
17th July 2012, 11:28 | #8 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
prefilter with removegrain(4).removegrain(11)
get vectors on prefilter mdegrain3 the original, with high thSAD (800 or so) get difference from mdegrain3 sieve difference by, again, removegrain(4).removegrain(11) apply sieved difference BTW, it seems pretty unclear what your original source actually is. The first screenshot is 800x533, the later ones are 1094x729. Varying resolution within one source is unlikely. None of those resolution is a standard res. In any case, process dotcrawl *without* any resizing beforehand.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) Last edited by Didée; 17th July 2012 at 11:34. |
17th July 2012, 11:36 | #9 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Thanks Emulgator and Didée. It looks like your ideas are really interesting, but unfortunately I really don't understand what to do. Would you give me an or several example(s) of your methods ? I don't ask for the "adjusted parameters" for my case of course but for one or several scripts to exemplify what you tell me to do.
EDIT : about my frames, it is just my mistakes, saving them unproperly, sorry. Last edited by sirt; 17th July 2012 at 11:41. |
17th July 2012, 11:43 | #10 | Link |
Big Bit Savings Now !
Join Date: Feb 2007
Location: close to the wall
Posts: 1,531
|
Keep in mind: This might (and most probably will) not work for your source !
Code:
#[*#%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start of DotCrawl Removal%%%%%%%%%%%%%%%%%%%%%%%%%%%%% [*#-------------------------------------------------------------Start of Defreq-FFT--------------------------------------------------------- Defreq(fx=64.0, dx=2.0, fy=16.0, dy=6.0, sharp=8.0, fx2=64.0, dx2=2.0, fy2=-48.0, dy2=6.0, sharp2=8.0, fx3=44.0, dx3=2.0, fy3=-0.0, dy3=6.0, sharp3=8.0, fx4=44.0, dx4=2.0, fy4=32.0, dy4=6.0, sharp4=8.0, show=0, info=true, plane=0) Defreq(fx=44.0, dx=2.0, fy=-32.0, dy=6.0, sharp=8.0, fx2=44.0, dx2=2.0, fy2=-64.0, dy2=6.0, sharp2=8.0, fx3=44.0, dx3=2.0, fy3=64.0, dy3=6.0, sharp3=8.0, fx4=64.0, dx4=2.0, fy4=-16.0, dy4=6.0, sharp4=8.0, show=0, info=true, plane=0) Defreq(fx=0.0, dx=3.0, fy=63.9, dy=4.0, sharp=8.0, fx2=0.0, dx2=8.0, fy2=0.0, dy2=3.0, sharp2=8.0, show=0, info=true, plane=1) Defreq(fx=0.0, dx=3.0, fy=63.9, dy=4.0, sharp=8.0, fx2=0.0, dx2=8.0, fy2=0.0, dy2=3.0, sharp2=8.0, show=0, info=true, plane=2) #Defreq nje rabotajet ! *]#-------------------------------------------------------------End of Defreq-FFT--------------------------------------------------------- #[*#---------------------------------------------------------------Start of Y-FFT----------------------------------------------------------- #Das folgende Y-Muster ist ein PAL-Beispiel, durch Dotcrawl verursacht, somit inhaltsabhänging #und gilt nur für Sektionen mit hoher Rot-Sättigung wie 113687, 119200-119300 bzw. Blausättigung wie Frame 92800-92900 SmoothCurve(interp=100, limiter=false, Ycurve="0-16;18-25;50-70;110-150;180-220;255-255", Ucurve="0-0;127-127;255-255", Vcurve="0-0;127-127;255-255", show=false)#Pre für 5x Y-F2Quiver) F2Quiver(nef=2, f1x1=176, f1y1=+60, f1x2=176, f1y2=+85, mf1=1.01, sh1=1, f2x1=176, f2y1=-50, f2x2=176, f2y2=-87, mf2=1.01, sh2=1) F2Quiver(nef=2, f1x1=176, f1y1=+165, f1x2=176, f1y2=+210, mf1=1.005, sh1=1, f2x1=176, f2y1=-165, f2x2=176, f2y2=-210, mf2=1.005, sh2=1) F2Quiver(nef=2, f1x1=232, f1y1=120, f1x2=232, f1y2=135, mf1=1.02, sh1=1, f2x1=232, f2y1=-120, f2x2=232, f2y2=-135, mf2=1.01, sh2=1) F2Quiver(nef=2, f1x1=232, f1y1=+8, f1x2=232, f1y2=-8, mf1=1.03, sh1=1, f2x1=232, f2y1=+245, f2x2=232, f2y2=+255, mf2=1.1, sh2=1) F2Quiver(nef=2, f1x1=90, f1y1=192, f1x2=110, f1y2=192, mf1=1.1, sh1=1, f2x1=90, f2y1=-192, f2x2=110, f2y2=-192, mf2=1.1, sh2=1) #SmoothCurve(interp=100, limiter=false, Ycurve="0-0;30-30;110-120;160-175;255-255", Ucurve="0-0;127-127;255-255", Vcurve="0-0;127-127;255-255", show=false)#für 4x Y-F2Quiver) SmoothCurve(show=false, interp=100, limiter=false, Ycurve="0-0;16-22;60-80;90-128;150-200;255-255", Ucurve="0-0;127-127;255-255", Vcurve="0-0;127-127;255-255")#Post für 5x Y-F2Quiver) #F2Quiver(test=true) #*]#---------------------------------------------------------------End of Y-FFT----------------------------------------------------------- #SmoothCurve(show=true, interp=100, limiter=false, Ycurve="0-0;63-63;127-127;150-150;191-191;255-255", Ucurve="0-0;255-255", Vcurve="0-0;255-255")#zur Kontrolle #[*#-------------------------------------------------------------Start of UV-FFT--------------------------------------------------------- SmoothTweak(hue1=+6, hue2=+6, saturation=1.05, show=false).F2QuiverUV("gp4", 0,255, degree=2)#gp4, degree=2 kompensiert die Ortsfrequenzen noch besser #SmoothTweak(hue1=+13, hue2=+13, saturation=1.1, show=false).F2QuiverUV("bp4", 0, 165, degree=1)#bp=4, degree=1 war fast genau so erfolgreich. #F2QuiverUV(test=true) #*]#---------------------------------------------------------------End of UV-FFT--------------------------------------------------------- #unbt() #*]#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%End of DotCrawl Removal%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Yes, and as Didée just said: No resizing beforehand !
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain) "Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..." Last edited by Emulgator; 17th July 2012 at 11:50. |
17th July 2012, 12:06 | #11 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
Here's the sieved MDegrain:
(Setup is made very slow on purpose.) (Also, without having the egg in my hand, I can't tell how hard you can press without breaking the egg.) Code:
YourSourceFilter("source.ext") ConvertToYV12() o=last pre = o.removegrain(4).merge(o,0.5).removegrain(4).removegrain(11) sup1 = pre.msuper() sup2 = o.msuper(levels=1) bvec3 = sup1.manalyse(isb=true, delta=3,blksize=16,overlap=8,search=5,searchparam=6,DCT=5) bvec2 = sup1.manalyse(isb=true, delta=2,blksize=16,overlap=8,search=5,searchparam=6,DCT=5) bvec1 = sup1.manalyse(isb=true, delta=1,blksize=16,overlap=8,search=5,searchparam=6,DCT=5) fvec1 = sup1.manalyse(isb=false,delta=1,blksize=16,overlap=8,search=5,searchparam=6,DCT=5) fvec2 = sup1.manalyse(isb=false,delta=2,blksize=16,overlap=8,search=5,searchparam=6,DCT=5) fvec3 = sup1.manalyse(isb=false,delta=3,blksize=16,overlap=8,search=5,searchparam=6,DCT=5) mdg = o.mdegrain3(sup2,bvec1,fvec1,bvec2,fvec2,bvec3,fvec3,thSAD=800) # drop to 600 or 400 ifyoucan ... raise to 1200 ifyoumust ... diff = mt_makediff(o,mdg,U=3,V=3) diff2 = mt_lutxy(diff,diff.removegrain(4).merge(diff,0.5).removegrain(4).removegrain(11), \ "x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?") mdg.mt_adddiff(diff2,U=2,V=2) stackhorizontal(o,last) return(last)
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
17th July 2012, 13:36 | #12 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Thank you both for the time spent to write this. Unfortunately, you are not lucky with me because it is still hard for to get what you have exactly done.
Emulgator : Let's say I want to use your script on a frame (with trim function for example). How can I ? Must I save it as an .avs file and try to apply it with : a=Emulgator() trim(0,1) + a+trim(2,3) + trim(3,0) Didee : My question is the same even though your script directly works on AVSP. |
17th July 2012, 13:44 | #13 | Link |
Big Bit Savings Now !
Join Date: Feb 2007
Location: close to the wall
Posts: 1,531
|
va=(Sourcefilter)
dedot=(EmuFFT....) firstframetodedot=100 lastframetodedot=200 Trim(va,0,frametoDeDot-1)++Trim(dedot,firstframetodedot,lastframetodedot)++Trim(va,lastframetodedot+1,0) #with the usual exception of the first and last frame...
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain) "Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..." |
17th July 2012, 14:46 | #14 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
BTW ... Is the source footage progressive, at least? All of a sudden, I got the fear that it might be hard telecined in fact, and you just manually selected some of the progressive frames for posting ...
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
17th July 2012, 15:32 | #15 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Didée : indeed, the source is an interlaced NTSC one ! I firstly iVTCed it without cropping either resizing it (I do have to do encoding tests with my compression firms's own h264 encoder on PROGRESSIVE stuff at that time) ; I guess that has an importance...
|
20th July 2012, 21:02 | #18 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Well I would rather prefer him helping me to ajust an existing filter or write a new one from zero ! Nothing works there whatever I tried:
- Dedot : zero - Tcomb : not enough for my taste - Lutdecrawl : zero - Checkmate : it deletes some dot crawl at least but blends unexpected areas Of course I maybe don't manage to set them correctly. So I suggest you jsaving one of the frames below and trying to remove this awful dot crawl ! It is terribly hard in my opinion.. |
20th July 2012, 21:22 | #20 | Link |
x264 fan
Join Date: Feb 2011
Location: In the trap
Posts: 458
|
Unfortunately it is NOT possible. Not only I would violate the rules but also the source is not my own property as I said ; I just have the right to test them, that's all. I think you can try to solve one of the frames below (check my second post), it would be the same.
|
Thread Tools | Search this Thread |
Display Modes | |
|
|