Identify specific artifacts in frame

[Tlen]

Hi,
I have a situation when randomly a frame is partly garbled with this artifact

https://images4.imagebam.com/dd/1e/7a/ME14VPYS_o.png

It's a very specific one in the form of rectangle / square "like interlaced". The stream is progressive.

Those rectangle can be in number of 5 to 10 scattered accross the frame.

In a 60 mins stream there are few frames with the artifacts, say less than 10, distributed randomly.

I would need a method to identify (just identify not fix) these affected frames (in analyzing pass)
and get a list of the affected frames.

Any ideas?

EDIT:
It could be some form of masking search method,
taking the Black & White image sample of the rectangular artifact and search for the image sample
it in every frame for (rough) matching.
But i'm not familiar with masking and frame scanning.

[Selur]

Sorry, I have no clue how to get a reliable list of frames or detect artifacts reliable.
Those look like data corruption during transfer.
(in which case, running the frame through ffplay might report errors on those frames)
Depending on their frequency and the rest of the content, spotless/despot/spotremover might work,..
Can't really say much without a sample.

[johnmeyer]

Here is a thread I started, many years ago, to detect individual bad frames which don't match either the previous or next frame:

Finding individual "bad" frames in video; save frame number; or repair (https://forum.doom9.org/showthread.php?t=174104)

It can work really well, but the quality of the script's results depend entirely on how "bad" the frame is. The script uses the AVISynth built-in YDifference functions. These sum up the differences in each pixel compared to the same pixel position in the previous or next frame. Those differences become massive at scene changes, but only looking one direction. However, when you have a corrupted frame, those differences are often really large in both directions, especially when you compare the metrics to those you get for nearby frames, which is what my script does.

You need to supply a sample, or you can just go ahead and use the script I posted in that thread (use the later one) and see what you get. If you have a simple background like your still image, the script might work really well. However, if those noise blocks happen during moving video, the metrics might not detect anything.

If you want to remove them, I would definitely suggest Spotless or RemoveDirtMC.

[StainlessS]

Spotless should fix those just fine (I think).

[johnmeyer]

Were these from DV or Digital8 tapes? I think I've seen this sort of thing before on those tapes. If you have another camcorder, you might try transferring the bad sections on that and see if the noise goes away, or moves to other frames.

[Tlen]

@johnmeyer
These are completely random transfer defects. They can be present in a sequence inside a static scene or 1 in 2000 frames in a moving one.
So the approach of comparing prev/next frame i don't think it can work in this case. Anyway I'll look into your thread.

The source is from a film strip scanning, and it's clear it was telecined, because in some parts (not many)
there are massive interlacing situation in scene changes where the two fields are just superimposed and not correctly sequenced
resulting in a crossfade effect, half fields from the previous scene and half fields from the new scene.
But this crossfade artifact pass almost unnoticed ('cause of speed and low numerosity).
It seems quite evidently that the transferring machine had some serious problem, because overrall the entire scan is correct and without any issue.
But randomly for no apparent reason something goes wrong and this squares appear for just 1 frame (it's very difficult to spot indeed).

The numerosity of this situations is very low, say 15 frames in 80000, for this reason I was searching a way to identify them and think what to do.

Any automatic approach would be overworked and could remove unneded things or low the overall quality unnecessary
(apart from these artifacts the scanning is flawless).

Anyway I'll look into despot too @StainlessS to see what we have there.
Thanks

[Frank62]

Looks like a (weak) Digital Betacam or HDCam problem.
It looks as if the problem is only on one field (I know the source is progressive), so I would try something like

Separatefields or bob to get both fields more similar
Subtract fields (selecteven from selectodd) and scan/log differences

[VoodooFX]

Any ideas?
But i'm not familiar with masking and frame scanning.

Cut and share few video snippets containing variations of these artifacts.

[Tlen]

@VoodooFX
x265 Lossless 25 frames of the situation of the first post.
The artifact is always the same (square with fixed dimension horizontal striped) everytime it appears
link (https://mega.nz/file/MQBWRaRR#dGnWqpkt7FstXqpxiACcuc3_bcz_ZqHw1goHA6X5pIs)

[poisondeathray]

What is the original format and compression ? There might be other ways to identify - e.g. mb quantizers etc...

[poisondeathray]

Another approach is to use comb detection with a low threshold. You will get more false positives, but preferred to missing a frame that has the artifact

eg.

WriteFileIf(last, "path\combedframes.txt", "IsCombed2(cthresh=5)", "current_frame")


You can run analysis pass print out txt file , and go back over the indicated frames with your eyes to verify

There are other comb detection algorithms and you can adjust the settings and threshold

[Tlen]

@poisondeathray
Original codec is AVC High@L4 BT.709

You suggest comb detection, but it doesn't seem to me a comb artifact, more like a static black stripes superimposed.
Your approach could work on scene change situations where indeed there is a combing artifact
(resulting in crossfade) but on such small square pieces probably I'll get so many false positive that it's not useful.
As i said we're talking about 1 in thousands frames.
BTW as I have some time I'll test, see what happens.

As for the quantizer approach, if the encoder "perceives" such a peculiar difference comparing to the context around it and
results in a spike (or whatever) in the quantizer sequence, pheraps it could effectively work (having the tool to spot such a spike).

[VoodooFX]

@VoodooFX
x265 Lossless 25 frames of the situation of the first post.
The artifact is always the same (square with fixed dimension horizontal striped) everytime it appears
link (https://mega.nz/file/MQBWRaRR#dGnWqpkt7FstXqpxiACcuc3_bcz_ZqHw1goHA6X5pIs)

Plugins:
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/
https://github.com/pinterf/TIVTC/releases

Script:

LWLibAvVideoSource("D:\tst.mkv")
WriteFileIf(last, "D:\tst_combedframes_list.txt", "IsCombedTIVTC", "current_frame")


tst_combedframes_list.txt:
12

[Tlen]

@VoodooFX
Worked like a charm. You're the man, guy!
Thank you so much!

It has been so magic that TIVTC got even a single frame end-credit (true combed) at almost all black level fadeout!

@poisondeathray
It seems in the end the combing approach was correct for TIVTC.
Nice catch!

Thank you all for your support!