How to remove broadcast errors monochromatic blocks

[spoRv]

I have a broadcast recording with those bad grey blocks, due to transmission errors; here you are an example:

https://s20.postimg.org/4hf5s04b1/hp4_glitch_example.jpg

This is the worst offender, that last for several frames, with some blocks difference in any frame; the "easy" one are just single blocks of different size, sometimes square, sometimes rectangular, that lasts one or two frames.

I did my homeworks, tried several filters, but any works.

A possible solution I thought is to paint those blocks (that are composed of different shades of gray) to a single color, and use exinpaint to erase them and replace with neighbour parts.

Tried with overlay and mask, but can't find a way to let those shades to be of a single hex value...

Any help would be greatly appreciated!

[Mounir]

the dvdropouts script might work here (old script )

avisource ("myvideo.avi")
morph(3422,3424) # to replace frame 3423


# Requires mvtools v1.11.4.5
function morph (clip c, int in, int "out", int "blksize")
{

Function fill_loop(string s, int stop, int count)
{
return (stop == 0) ? s : string("tofill.mvflowinter(fill_vb,fill_vf,time=" + string(100*(count - stop)/float(count))) + ",thscd1=255,thscd2=255).selectevery(tofill.framecount(),0)," + fill_loop(s,stop-1,count)
}

out=default(out, 0)
blksize=default(blksize,16)

d=c.trim(in,out)# in-1?

numframes=d.framecount-2
tofill=d.selectevery(c.framecount(),0,c.framecount()-1)

fill_vf=tofill.mvanalyse(truemotion=true,blksize=blksize,pel=2,isb=false,overlap=blksize/2)
fill_vb=tofill.mvanalyse(truemotion=true,blksize=blksize,pel=2,isb=true,overlap=blksize/2)
filled=eval("interleave(" + fill_loop("" ,d.framecount()-1,d.framecount()-1) + "tofill.selectevery(tofill.framecount(),tofill.framecount())).assumefps(c.framerate())")
c.trim(0,in-1)++filled++c.trim(out+1,0)
}

[spoRv]

Tested, doesn't work... :(

[FredThompson]

I've been thinking about similar issues with OTA dropouts. Other problems can be pixellation, "freezing" of the video, and audio pops from a large abrupt change in teh audio wave. I don't think any of the stream health analysis methods will work because the issue is the source material, not the stream which was created from it.

[manono]

Although you'd have to find and fix them individually, unless there's too much movement from frame to frame, frame interpolation should work. Maybe provide a short ten-second sample so we could have a look.

[johnmeyer]

I agree with manono: if you can detect frames that contain a glitch (perhaps a YDifference metric?), you can then replace the entire frame with one interpolated from its neighbors.

Another approach that might work is to use the motion-compensated version of RemoveDirt. I posted several ways to use this over in the two epically-long VideoFred film restoration clips (search for "RemoveDirtMC"). I would need a sample to know it is might work, however, because the one single freeze frame posted above makes the defect look to be fairly low contrast. RemoveDirt works best when the "glob" (highly technical term :)) stands out starkly against the background, which is typical of dirt on film emulsion.

So, post a sample ...

P.S. Here is code that uses YDifference to detect a frame that is different from both of its neighbors. This sort of test useful for detecting single-frame glitches, blank frames, flash frames (from a photographer's flash), etc. This particular script merely creates a file with a list of frame numbers. You can replace that WriteFile code with a call to FillDrops (a script I wrote using MugFunky's old code).

#This script detects single bad frames

global diffhreshold = 10
filename = "e:\JHM_Scenes.txt"
source=AVISource("e:\fs.avi").convertTOYV12().killaudio()

# Use the following code block to determine the proper diffthreshold
# Comment out this block and enable the writefile block to produce a list of frames that should be replaced
script = """Subtitle("\nprevious = " + String( YDifferenceFromPrevious(source)) + \
"\nnext = " + String( YDifferenceToNext(source) ), lsp=0)"""
final=Scriptclip(source, script)
return final


#WriteFileIf(source, filename, "(YDifferenceFromPrevious(source) - YDifferenceToNext(source)> diffhreshold)", "current_frame-1", append = false)

Here is some code that will replace a frame with one that is interpolated from its neighbors. There is no scene detection logic, so it will do weird things if applied to a frame adjacent to a scene change:

function createframe (clip c)
{
super=MSuper(c,pel=2)
vfe=manalyse(super,truemotion=true,isb=false,delta=1)
vbe=manalyse(super,truemotion=true,isb=true,delta=1)
final = mflowinter(c,super,vbe,vfe,time=50)
return final}

[spoRv]

John, thanks for your scripts, they are always interesting.

But, as I wrote in the first post, that nasty artefacts found in the example lasts for several frames (five or six, can't remember now), hence an interpolation script could at least be useful to delete the first and the last, but would make a mess trying to replace all the affected frames with interpolated ones.

That's why I thought about the exinpaint idea; finding the affected area and replace it with a single color would create a mask for exinpaint; albeit far from perfect, in the dark parts of the shots with those blocks - like the one found in the example - the script would simply fill that area with black surrounding parts, surely better than "in your face" grey blocks, I think.

The bad frames detector seems nice, it could be very helpful, I'll try it ASAP!

:thanks:

[poisondeathray]

exinpaint (or any inpainting) usually creates more problems because of frame to frame fluctuations => flicker . You usually need additional temporal smoothing to make it usuable

There are interpolation scripts that can do ranges like your 5 or 6 . If you combine that with masks it would be more accurate / fewer artifacts

Were you thinking of manually creating masks ? What is the pattern of the defect ?

[johnmeyer]

If the defects stay in the same x-y location, and are about the same size and shape for several frames in a row, then they will be tough to fix, no matter what method you try.

For paid work, I often put two copies of the video onto the timeline in my NLE (Vegas), offset one copy by a few frames, and then put a mask over the defect so the offset video shows through from the other track. If the camera is not moving, this can produce excellent results. Feathering the mask edges is an important element in getting this to look good.

It is, of course, a lot of work.

[GMJCZP]

spoRv, Try this home remedy that I developed with ALL the help of Didée:

Here (https://forum.doom9.org/showthread.php?p=1567932#post1567932)

¡Good luck!

Edit: maybe you should modify the function a bit to the resolution of the video.

[manono]

that nasty artefacts found in the example lasts for several frames (five or six, can't remember now), hence an interpolation script could at least be useful to delete the first and the last, but would make a mess trying to replace all the affected frames with interpolated ones.
That remains to be seen. You can interpolate parts of frames and leave the rest of them alone. Again, though, a video sample is worth a thousand words.

In any event, as pdr said, inpainting isn't nearly as good (usually) as interpolating, perhaps combined with masks.

[spoRv]

Here you are the link (https://fs06n1.sendspace.com/dl/9daae0934ce9bb9406eb853d09b61d53/592b95e62ac7529b/qyo9t0/glitch_sample.avi) to the sample clip (8MB Lagarith); the glitch lasts for 12 frames! :scared:

I repeat, this is the worst offender, but if there is a solution that would remove it, it certainly would remove also smaller ones, that last for fewer frames, right?

[manono]

I repeat, this is the worst offender, but if there is a solution that would remove it, it certainly would remove also smaller ones, that last for fewer frames, right?
It depends on what's going on around it. If the frame is complex enough (lots of movement), even one frame might not interpolate well, if you're picky about the results. Your 12 frame artifact is a good example. The first result (Glitch1.mp4) might not be good enough for you. So I made another one by replacing parts of the frame which should be good enough. The first script was:

AviSource("glitch_sample.avi")
A=Last
B=ReplaceFramesMC(7,12)
B=B.Crop(560,750,-1150,0)
Overlay(A,B,560,750)

The second version (Glitch2.mp4) replaced parts of the frames by using this (by replacing the artifacts with a good version from a different frame):

Q=ImageSource("7.bmp",End=7).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="7")
Q=ImageSource("8.bmp",End=8).ConvertToYV12(
ReplaceFramesSimple(Last,Q,Mappings="8")
Q=ImageSource("9.bmp",End=9).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="9")
Q=ImageSource("10.bmp",End=10).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="10")
Q=ImageSource("11.bmp",End=11).ConvertToYV12(
ReplaceFramesSimple(Last,Q,Mappings="11")
Q=ImageSource("12.bmp",End=12).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="12")
Q=ImageSource("13.bmp",End=13).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="13")
Q=ImageSource("14.bmp",End=14).ConvertToYV12(
ReplaceFramesSimple(Last,Q,Mappings="14"
Q=ImageSource("15.bmp",End=15).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="15")
Q=ImageSource("16.bmp",End=16).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="16")
Q=ImageSource("17.bmp",End=17).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="17")
Q=ImageSource("18.bmp",End=18).ConvertToYV12()
ReplaceFramesSimple(Last,Q,Mappings="18")

By being more careful I might have been able to do a better job. And I'm sure pdr can do better yet since some of what I know I learned from him.

If you need filters or more information, just ask.

https://www.sendspace.com/file/achybl

https://www.sendspace.com/file/pclm2b

[spoRv]

manono, thanks a lot for your hard work!

Even first solution is good enough; so, you interpolated the missing frames, and used them to create a "patch" to apply over the glitches, right? Nice solution!

I wonder if there could be an automated script that could do that! ;)

[GMJCZP]

spoRv, did you try my suggestion?

[manono]

...so, you interpolated the missing frames, and used them to create a "patch" to apply over the glitches, right?
I interpolated just around the artifacts. That's what the crop does - shrink down to the area needing fixing so the rest of the frame isn't affected. It can also be done using a mask to isolate the problem area.

The second part of the script did create a patch from frame number 19, which then was applied over the 12 frames, one by one. I could have just skipped interpolating the 12 frames and gone right to the patch job, but I think in most cases you'll be able to interpolate successfully. As you said, your example was the worst of the bunch. Better yet, of course, would be to not have those artifacts to begin with.

[spoRv]

GMJCZP, nope, as manono was so kind to have solved the problem for me! ;)