Masking AVCHD compression artifacts

[Efenstor]

I thought, is there a way to mask AVCHD compression artifacts, say, with grain? Like this:

http://www.efenstor.net/external/1193_orig.jpg
http://www.efenstor.net/external/1193_grain.jpg

Or, even better, to mask only badly-compresed areas of the frame. I tried MMask (MVTools) and MaskTools to mask motion areas but the results are very inaccurate (huge artifacts are sometimes missed), because, in fact, actual bitrate should be analyzed for creation of such a kind of mask. May be DGDecNV could produce some kind of a guide file which later would be used for mask creation?

[MadRat]

Are you sure that's compression artifacts? It looks like interlace artifacts to me.

[thetoof]

How did you capture this???

Lossless or highly compressed to destroy the image and even introduce blocking?

[Efenstor]

It was shot with a Canon HF100 camera, in AVCHD with 17 mbps data rate, in telecined 24p (the only choice in my camera). I used DGDecNV to decode it and TFM (mode=4) + TDecimate (mode=1) to re-create the original 24p frames. I tried manual IVTC but it produces identical results. Such artifacts are often seen in shaky footage with a lot of complex textures (branches, grass, autumn leaves, etc.). Before that particular frame there was a huge semi-transparent lens flare covering the right part of the frame. As to why there's no blocking, it's just the NVidia's deblocking algorithm is on. At 17 mbps (it's the maximum bitrate for my camcorder) some detail loss can be noticed even in moderately shaky footage, may be I'm a perfectionist but I think it's a good idea to mask such artifacts with grain, it's more natural for an eye to see some noise instead of distortions, moreover it shouldn't look unnatural as there's a lot of high motion.

[thetoof]

Hmh, well this sucks... I'm actually working right now on a HDV project that doesn't show these kind of artifacts (captured via Premiere Pro @ 25mbps mpeg-2). I'm no AVCHD pro (at all), but it's weird that any interlacing artifacts remain after IVTC on a supposedly HQ source... Wouldn't a simple killcomb function like vinverse do the trick without adding tons of grain? Or some other deinterlacing method (slowest being bobbing the original + tdecimate(1,3,5))?

Not really recommending anything, simply asking as I'm curious to know why such a flaw exists... and of course an efficient way of solving it as I tend to use overly slow scripts...

[Efenstor]

Actually the ITVC artifacts don't bother me as much as the deblocking artifacts, those unnaturally blurred areas blatantly show the very nature of H264 compression, just where it shouldn't be really seen. I'd really love to mask them with grain, to simulate film. Of course, I can try to save about $2000 and finally buy a semi-pro camera but does it really worth it, just because of some artifacts? May be MaskTools + MVTools still can help?

[Blue_MiSfit]

If you have blocking, DFTTest is a great way to fix things.

Post a sample. You shouldn't be seeing interlacing artifacts after IVTC. My thinking is that this is being done incorrectly

~MiSfit

[poisondeathray]

If you have blocking, DFTTest is a great way to fix things.

Post a sample. You shouldn't be seeing interlacing artifacts after IVTC. My thinking is that this is being done incorrectly

~MiSfit

He's not complaining of blocking, he's seeing "deblocking" artifacts or oversmoothing

Overcompression with AVCHD usually manifests as oversmoothing detail; with HDV, it usually manifests as macroblocking

You could use a better consumer model. Some canon AVCHD camcorders go up to 24Mb/s and I think it would help specifically in this scenario

Also shooting technique with stabilization would help e.g. using tripod, harnesses, or steadicam etc... , and would minimize shaking and help with compression

The remenant IVTC artifacts are a different matter, and I agree maybe a native video sample would help

[Efenstor]

My thinking is that this is being done incorrectly

Nope, I'm pretty sure it has been encoded incorrectly, it's the flaw of the camcorder software.

I've made the following script to reduce the interlacing artifacts (only in softer areas), and, to my surprise, it even brought up some additional details:

TFM(mode=4)
TDecimate(mode=1)
m = mt_edge().mt_expand().mt_invert()
o = yadif()
Overlay(o, mask=m)


http://www.efenstor.net/external/1193_fixed.jpg

Still combing on edges remains and I guess there's no detail-safe solution for this yet, even nnedi2 is always worse than true IVTC.

poisondeathray, thanks for the ideas. I have a rather good steadycam-like thing made by B-Hague but it's not that easy to use on the go in an off-road trip.

[Efenstor]

As for the sample, try this (old) one: http://www.efenstor.net/external/00066.MTS
Both kinds of the artifacts are present there.

[Blue_MiSfit]

Taking a look here..

[Efenstor]

I already have a better idea how to fix the interlacing artifacts. Since both fields represent the same frame, only differently compressed, I'll try Bob+MaskTools to create the mask from the field difference, then Yadif or nnedi2 what's different, identical areas (i.e. well compressed details) will be kept intact.

[Efenstor]

I already have a better idea how to fix the interlacing artifacts. Since both fields represent the same frame, only differently compressed, I'll try Bob+MaskTools to create the mask from the field difference, then Yadif or nnedi2 what's different, identical areas (i.e. well compressed details) will be kept intact.

Nothing useful has come out of that. Here's the final version yet:


ReduceArtifacts(70)

function ReduceArtifacts(source, th)
{
m1 = mt_edge(source).mt_expand().mt_invert()
m2 = mt_motion(source, thT=255, thY1=th, thY2=th, thC1=th, thC2=th)
m = mt_logic(m1, m2, "or")
Overlay(source, Yadif(source), mask=m)
}

And the result:
http://www.efenstor.net/external/1193_new.jpg

Pretty good, but still some artifacts on the edges remain. Of course, the threshold can be lowered and they will be gone, but then details will go too.

Another sample frame that I used to check the function for detail-safety (source and processed):
http://www.efenstor.net/external/2389.jpg
http://www.efenstor.net/external/2389_new.jpg

[Efenstor]

Getting back to my old thread. I want to share my latest script, very simple but fine-tuned, that I use to pulldown 24p footage made with the early clumsy Canon AVCHD chips, e.g. Canon HF100 (yes, I still have no better camera). The difference from just a pulldown is the post-processing part that very carefully removes most of the residual combing artifacts.

Requirements:
AviSynth MT (http://forum.doom9.org/showthread.php?t=144852)
TIVTC (http://web.missouri.edu/~kes25c)
LeakKernelBob (http://neuron2.net/kerneldeint/kerneldeint.html)
mt_masktools v2.x and MaskTools v1.5.8 (http://manao4.free.fr/)

#===============================================================================
# Pulls down a telecined 24p frame
# s: the source clip
# threads: number of processing threads (cores)
# eth: edge masking threshold (25 is optimal for Canon HF100 24p)
# cth: comb masking threshold (15 is optimal for Canon HF100 24p)
#===============================================================================
function Pulldown(clip s, int threads, int eth, int cth)
{
s = s.TFM().TDecimate(mode=2, rate=23.976, m2PA=true)
di = MT(s, """LeakKernelBob(1,0,false,false)""", threads=threads, overlap=16)
di = MT(di, """Overlay(SelectEven(), SelectOdd())""", threads=threads, overlap=0)
em = s.mt_edge("prewitt", eth,eth,eth,eth).mt_expand(255,255,"vertical").Invert()
cm = s.CombMask(cth,cth).mt_inpand(255,255,"vertical")
m = mt_logic(cm,em,"or")
mt_merge(s,di,m)
}

If you want it single-threaded, then you need to remove all the MT(...) stuff, retaining only strings inside the triple quotes.

Here is the worst-case example. Simple pulldown (TFM+TDecimate) VS my simple script:

http://www.efenstor.net/external/pulldown_cr_cr.jpghttp://www.efenstor.net/external/pulldown_proc_cr_cr.jpg

And the "best case" example, showing how careful it is with the fine detail (the camera is panning vertically here):

http://www.efenstor.net/external/pulldown2_cr.jpghttp://www.efenstor.net/external/pulldown2_proc_cr.jpg

You can trust me that the best deinterlacing+decimation, pulldown+deinterlacing or just CombMask alone can't do that. I tried, of course.