Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
|
31st May 2010, 06:37 | #1 | Link |
Registered User
Join Date: Nov 2006
Posts: 19
|
What is this artifacting called, and can anything be done about it?
Greetings all,
Delurking after 6+ years as there's finally an issue I can't remedy by searching alone. I'd be grateful for any assistance that anyone cares to offer. I'm cleaning up some DVDs from a 1960s Eastern European TV show. A few episodes have a terrible "corduroy" look to them, as you can see below: What the heck is that, and is there anything that can be done about it? Even some search terms would be helpful. I've thrown a few denoisers at it (TemporalDegrain, Neat Video), since they were already part of my workflow. They helped a little but clearly aren't meant for this kind of thing. I've also considered an interference frequency filter (DeFreq or FanFilter) but my preliminary tests weren't fruitful. If it makes any difference, I believe this was originally shot on (24fps) film and then somehow frameblended to (25fps) SECAM (or a PAL master). From there, it seems, some musty old master tape was used to create the PAL DVDs. (That's my guess, anyway, after spending hours and hours on various other artifacts.) Finally, if anyone's really interested, a 2-second VOB extract is available here: http://www.megaupload.com/?d=B5O5UK3B (1.5MB) Thanks! |
31st May 2010, 14:45 | #2 | Link |
Registered User
Join Date: Sep 2007
Location: Europe
Posts: 602
|
I think this is analog video noise which has been "thickened" by low-pass filtering and sharpening. The noise is probably the result of an old telecine, build-up from copying to analog tape formats... it may even be fine film grain that has again somehow taken on this "thick" look - or a combination of all of the above.
When I was faced with this problem, NeatVideo (as you say) couldn't do much for it. In the end, I found this script useful (it's slow, but worth it): Code:
source = last pred = source # to get stronger denoising, put denoisers here, they will change how motion vectors are predicted backward_vec2 = pred.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true) backward_vec1 = pred.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true) forward_vec1 = pred.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true) forward_vec2 = pred.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1, truemotion=true) maskp1 = mvmask(kind=1, vectors=forward_vec1, ysc=255).UtoY() maskp2 = mvmask(kind=1, vectors=forward_vec2).UtoY() maskp3 = mvmask(kind=1, vectors=backward_vec1, ysc=255).UtoY() maskp4 = mvmask(kind=1, vectors=backward_vec2).UtoY() maskf = average(maskp1, 0.25, maskp2, 0.25, maskp3, 0.25, maskp4, 0.25).spline36resize(source.width, source.height) smooth = pred.fft3dfilter(bw=16, bh=16, ow=8, oh=8, bt=1, sigma=4, plane=0) source2 = maskedmerge(source, smooth, maskf) source3 = source2.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=2) source3 ttempsmooth(maxr=7) gradfun2db(1.51) Last edited by Lyris; 31st May 2010 at 14:48. |
31st May 2010, 16:37 | #3 | Link |
Registered User
Join Date: Feb 2004
Posts: 743
|
Uploaded with ImageShack.us Simple one line of code, You may want to add SeeSaw or LSF AffineCurvatureFlow(2) Richad photorecall.net |
31st May 2010, 16:58 | #5 | Link |
Registered User
Join Date: Feb 2004
Posts: 743
|
|
31st May 2010, 17:25 | #6 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
Hmmh, well ... it does dimish the artifacts. But when I do a zoom-in, it seems that it's removing more of the detail then of the artifacts. (I don't need to show that, zooming can everybody do himself.)
Short of using {a certain proprietary processing tool} ... is someone used to setting-up and using FFT3DFilter's noise pattern mode? This seems like a typical application case for that.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
31st May 2010, 22:05 | #7 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,377
|
apologies for the "proprietary tool"
After Effects, box blur, 1.5 strength, 1 iteration, horizontal only It's a little blurry, but at least it gets rid of most of the vertical lines. No denoising or sharpening applied (which you might want to do after) I looked for the equivalent function in vdub which has a box blur, but it doesn't have vertical/horizontal control, and it seems to "overdo" it (doesn't have finer gradations). neuron2 also has the equivalent avisynth plugin from vdub, but I didn't test it. Maybe it has finer control? Or maybe one of the gurus can modify it? http://neuron2.net/boxblur/boxblur.html Last edited by poisondeathray; 31st May 2010 at 22:11. |
1st June 2010, 02:21 | #8 | Link |
Registered User
Join Date: Nov 2006
Posts: 19
|
Thank you Lyris, rfmmars, Didée and poisondeathray for your suggestions thus far.
And thanks also, neuron2, for taking an interest.
Lyris: That script actually does very little to the corduroy artifact in this material. I even tried throwing in a turnright() and turnleft() at the beginning and end, respectively, in hopes that gradfun2db would respond better to horizontal lines. Doing so yielded only the tiniest improvement, however. Here's a screenshot (with even an additional denoiser added to line two [pred = source.mc_spuds(mode="very high")] because I'm a masochist): rfmmars: Interesting function--thanks for making me aware of it. ("New smoothing plugin: Curvature Flow Approximation" by AMSS0815, if anyone's interested.) I tried numerous settings in addition to your suggestion. Ultimately, though, I wasn't happy with its ability to only partially alleviate the corduroy, particularly at the price of significant detail loss. Didée: Unfortunately, my experiments with FFT3DFilter's noise pattern mode were not promising. Here are the results, using fft3dfilter(pframe=44787,pfactor=1,px=26,py=16,wintype=2) [44787 is both the frame from which the noise pattern is derived and the frame in the screenshot; the black box in the screenshot shows the location + size of the noise pattern source): poisondeathray: Looks promising--I almost forgot about After Effects! I'll try it out tonight... |
1st June 2010, 03:21 | #9 | Link |
Registered User
Join Date: Dec 2003
Location: MO, US
Posts: 999
|
kurish, can you post another video sample (maybe from around frame 44787 as shown in your last post)? In the one in your first post it is hard to tell what should be flat background (can't tell if the walls should be or if they are textured).
|
1st June 2010, 03:48 | #10 | Link |
Registered User
Join Date: Nov 2006
Posts: 19
|
Gladly!
3 second clip: http://www.megaupload.com/?d=MTV5OA3V The frame that's being screenshotted so much here is #41 in that file. FYI, the DVDs (and vob clips) have a (superfluous) chroma component. I'm nuking it with greyscale(). Thanks! |
1st June 2010, 07:24 | #11 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
Big Blur Sledgehammer contest?
Using a simple Gauss/Median combination (MinBlur function) ... Using MinBlur plus Median ContraSharp
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
1st June 2010, 18:03 | #13 | Link |
Registered User
Join Date: Dec 2003
Location: MO, US
Posts: 999
|
dfttest results on the second clip using nfile to estimate noise spectrum with various over subtraction factors (3.0,5.0,7.0):
hotel_dfttest_3.0.avi hotel_dfttest_5.0.avi hotel_dfttest_7.0.avi I used sbsize=20/sosize=16, but window size didn't make any real difference (also tried 16,24). Everything else was defaults. Would probably want to sharpen it afterward, but I didn't try any further processing (just wanted to see how dfttest's noise spectrum estimation worked in this case). nfile was: a=x.x 35,0,314,414 42,0,314,414 62,0,314,414 54,0,88,488 70,0,88,488 30,0,20,666 48,0,20,666 58,0,390,410 32,0,76,446 |
1st June 2010, 18:17 | #14 | Link |
Registered User
Join Date: Mar 2009
Location: Germany
Posts: 5,769
|
What do you intend to do with the movies?
Watching them on a PC or on a TV? Because the processed images that were uploaded are OK for a PC (which is sharp by default) but "flat" on a CRT TV (which is a bit blurry by default). For a CRT TV I would reduce the noise a bit less. For an LCD HDready/FullHD TV I would soften more (more agressive), as they tend to emphase the edges, thus the noise. The noise I think is the film noise which film was TCed on tape (which added some noise of its own), then "flattened" by the MPEG-2 algorithm (see the second image). |
1st June 2010, 21:00 | #15 | Link |
Registered User
Join Date: Feb 2004
Posts: 1,348
|
Here's my entry.
Code:
FIR = Mt_Convolution(Horizontal=" 48 64 96 64 48 ", vertical ="1", u=-128, v=-128) Diff = Mt_Makediff(Last, FIR) THR=string("121") MedianDiff = MT_Luts(Diff, Diff, mode="med", pixels = " 0 0 1 0 2 0 -1 0 -2 0 " , expr = " X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1) ReconstructedMedian = mt_makediff(Diff, MedianDiff) Mt_AddDiff(FIR, ReconstructedMedian) |
1st June 2010, 23:24 | #16 | Link |
Registered User
Join Date: Nov 2006
Posts: 19
|
tritical: Damn!! Love it. Thank you. [Edit: Removing a sentence here--I read the dfttest doc and figured it out. ]
Screenshots from your avis, for the record: *.mp4 guy: Awesome, as well! Thank you, too. I have some experimenting to do between your approach and tritical's--It's great to have options. Screenshots follow--first set is nothing but your script, second set appends the following: Code:
source = last denoised = source.minblur(2).fluxsmootht() temporaldegrain_chromamod(source, denoised, degrain=3, chroma=false) *.mp4 guy code: *.mp4 guy + temporaldegrain: Ghitulescu: They'll be watched on an LCD HDTV. I appreciate your helpful input. I'll take it into consideration when creating the final encodes. Last edited by kurish; 2nd June 2010 at 01:38. Reason: I R'd TFM |
2nd June 2010, 01:21 | #17 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
@mp4guy: Very nice isolation of the target. Hat off, I like it.
kurish - MDegrain after mp4guy's cleaner script is what I would've tried, too. TemporalDegrain is same blood, just darker. Though, for my personal taste, you have it a little too strong - reducing to degrain=2 probably would make things look more natural. Surfaces' textures are rather weak in the source to start with, so I'd vote for leaving a little more in the whole thing. But then, that's personal taste ... if you like it smooth, then make it smooth.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
2nd June 2010, 03:30 | #19 | Link |
Registered User
Join Date: Feb 2004
Posts: 1,348
|
Thanks.
However I wasn't quite satisfied. New version below should give essentially perfect separation. Code:
FIR = Mt_Convolution(Horizontal=" 48 64 96 64 48 ", vertical ="1", u=-128, v=-128) Diff = Mt_Makediff(Last, FIR) THR=string("256") MedianDiff = MT_Luts(Diff, Diff, mode="med", pixels = " 0 0 1 0 2 0 -1 0 -2 0 " , expr = " X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1).fft3dfilter(bw=2, bh=16, ow=1, oh=8, bt=1, sigma=16, sigma2=0.75, sigma3=16, sigma4=16, plane=4) ReconstructedMedian = mt_makediff(Diff, MedianDiff) Mt_AddDiff(FIR, ReconstructedMedian) |
2nd June 2010, 23:38 | #20 | Link |
Registered User
Join Date: Nov 2006
Posts: 19
|
|
Tags |
corduroy, ridge |
|
|