Remove cross hatch pattern from Kinescope

[johnmeyer]

I received a DVD from a client who wants me to restore a Kinescope that has a nasty cross-hatch pattern. There is also considerable macroblocking from the low-bitrate used to encode 2+ hours onto a single DVD. I am less concerned about that issue, at least for the moment. Here is a snapshot, showing the problem:



And here is a short clip, cut directly from the DVD VOB file, using Womble's MPEG Video Wizard:

https://www.mediafire.com/?x0hvwpw0vgmiw2x

The person who encoded the DVD did so at 23.976 (most of these amateurs don't do that, so at least this got done correctly).

I have tried applying some Gaussian blur, but it does too much damage to the detail. I was wondering if anyone has other ideas?

Don't worry about making the watermark look strange because I can mask that or attempt to remove it with Delogo. Also, don't worry about the flicker, because I know how to deal with that.

Thanks!

[raffriff42]

Severe cross chroma interference (dot crawl). Notice there are no dots on the white uniforms, but the there very high amplitude dots on the green turf.

*The dots are coarser than the legitimate detail, so they can't be blurred out.
*If they are from chroma as I suspect, they're not in a regular pattern, so they can't be cancelled out with an opposing pattern.
*The dots are beating against the pixel resolution, which complicates things.

Best solution, get a new transfer, this is junk.

EDIT ...or Gaussian blur, radius 1.3 + Sharpen

[johnmeyer]

Originally Posted by raffriff42

Severe cross chroma interference (dot crawl). Notice there are no dots on the white uniforms, but the there very high amplitude dots on the green turf.

*The dots are coarser than the legitimate detail, so they can't be blurred out.
*If they are from chroma as I suspect, they're not in a regular pattern, so they can't be cancelled out with an opposing pattern.
*The dots are beating against the pixel resolution, which complicates things.

Best solution, get a new transfer, this is junk.

EDIT ...or Gaussian blur, radius 1.3

Very useful and interesting insight about the problem being caused by dot crawl. You are definitely correct about the lack of dots in the brighter areas of the picture. The game was broadcast in 1976, so it was definitely in color. This is a Kinescope of that broadcast, something that was seldom done after the mid-1960s, but was still sometimes done for advertisers, so they could confirm that their ads ran at the correct time, and that the ad wasn't truncated or ruined in any way. In fact, this is almost certainly why this Kinescope was created, because it includes all the original commercials, just as they aired.

However, since there is no videotape of this game, there is no possibility of getting a better transfer. Other than encoding to DVD, this IS the original and only transfer, and this problem was probably introduced by the person running the Kinescope equipment.

I'll keep playing around with various blur settings, and also see if I can come up with something that responds more to diagonal lines. You are correct that the pattern is not spatially fixed, but the one thing that IS consistent is the angle of the cross hatch pattern. If there is any hope of doing something more intelligent than a simple blur, I think that exploiting that fixed angle may be part of the magic. Convolution?

[raffriff42]

The angle is due to the chroma phase inverting on alternate lines. (It's NTSC, if that wasn't obvious from the American football)

Turns out it's pretty recoverable. Gaussian blur works better here than regular blur.

# http://avisynth.nl/index.php/Variableblur
LoadPlugin(path + "variableblur07\variableblur.dll") 
ImageSource("E:\Data\Downloads\cross-hatch1.png")
ConvertToYV12
GaussianBlur(varY=Pow(1.2, 2.0), U=-128, V=-128) ## "blur radius" is aprox. "varY" squared
Unsharp(varY=1.0, strength=1.0, U=-128, V=-128)

If I'm not mistaken, people have ruminated here in the past on how to recover the color information from the dots! Don't know if they got anywhere!

[wonkey_monkey]

Bah. In the time it's taken me to write an interesting essay on the history of colour recovery, you seem to have covered all the main points between the two of you!

Anyway, I found that the following kernel may be better than a Gaussian blur:

0 1 0
1 0 1
0 1 0
____
4

It seems to get rid of the crosshatching just as effectively, but the result is much less blurred, and can then be sharpened (some residual crosshatching is still present).

Edit: or this one with built-in sharpening:

-1  2 -1
 2  0  2
-1  2 -1
_____
4

If I'm not mistaken, people have ruminated here in the past on how to recover the color information from the dots! Don't know if they got anywhere!

I don't know about the people here, but the boffins at the BBC have used it to restore "lost" episodes of Dad's Army and Doctor Who to full colour from black and white film recordings.

[johnmeyer]

Originally Posted by raffriff42

If I'm not mistaken, people have ruminated here in the past on how to recover the color information from the dots! Don't know if they got anywhere!

Ah yes, I remember reading about that:

Colour recovery from chroma crawl

The BBC apparently was able to recover some color in "Dr. Who" episodes using this trick. I'm not sure exactly how it was done. It would be quite a trick to turn THIS into a color broadcast. Wow, would that ever surprise my client!

I'm working on two projects simultaneously, so I'll try your blur code in a few hours when I get back to this project. Thanks!

[edit]I clicked on the B&W pic in that Wikipedia example, and sure enough, the cross hatch pattern is identical to my film/video. So, excellent deduction, raffriff42!

[edit2]Simultaneous posting with davidhorman. Thanks for the convolution matrix. I was playing with that in Sony Vegas, but not getting the results I wanted. Vegas also lets you rotate the matrix before applying it, something that might be useful for this problem.

[Reel.Deel]

Originally Posted by johnmeyer

If there is any hope of doing something more intelligent than a simple blur, I think that exploiting that fixed angle may be part of the magic. Convolution?

You can try V. C. Mohan's frequency domain filters (FFTQuiver and FQuiver). I don't know how well it will work since the video has lots of blocking, compromising the fixed pattern in those areas.

Originally Posted by johnmeyer

And here is a short clip, cut directly from the DVD VOB file, using Womble's MPEG Video Wizard:

Does it cut the video losslessly? The bitrate seems a bit too low, hence all the blocking.

[wonkey_monkey]

Originally Posted by johnmeyer

Thanks for the convolution matrix. I was playing with that in Sony Vegas, but not getting the results I wanted. Vegas also lets you rotate the matrix before applying it, something that might be useful for this problem.

I edited my post to add another (just in case you missed it), which has built-in sharpening so the result is almost indistinguishable in sharpness from the original. No rotation needed.

[johnmeyer]

Wow, what a lot of great help. The convolution approach, as I mused at the end of my initial post, seems to produce the best results so far. I wish I could find a good tutorial on convolution settings, because I "sort of" understand it (it combines all pixels immediately adjacent, using the weightings specified), but never could find a precise algorithmic description.

I used the convolution fX in Sony Vegas, rather than AVISynth simply because it was easier for me. I'm pretty sure it uses the exact same algorithm. Here is the result for one frame:

Uncorrected


Blurred using first davidhorman matrix


Blurred and sharpened using second davidhorman matrix


The blur alone seems to do a pretty good job.

[wonkey_monkey]

(it combines all pixels immediately adjacent, using the weightings specified), but never could find a precise algorithmic description.

That is pretty much it. My first matrix, for example, just averages the north, south, east and west neighbours.

I think the second matrix does a much better job - just look at the logo to see how much sharper it is.

I also found that mdegrain1 (from MVTools 2) subsequently does a pretty good job of getting rid of the flicker.

[johnmeyer]

Originally Posted by davidhorman

That is pretty much it. My first matrix, for example, just averages the north, south, east and west neighbours.

I think the second matrix does a much better job - just look at the logo to see how much sharper it is.

I also found that mdegrain1 (from MVTools 2) subsequently does a pretty good job of getting rid of the flicker.

Yes the logo is sharper, but I'm probably going to remove it, or at least see if it looks better with it removed. The client I am working for purchased this disc from that company, but they delivered it with the logo. I don't mind protecting copyrighted material, but a little "bug" in the corner would have sufficed.

As for MDegrain, I do plan to do quite a bit of restoration after I remove the cross hatch. I'll try to reduce the macroblocking that resulted from poor MPEG-2 encoding at low bitrates; the film dirt; the general noise; and the flicker. For films like this, I find that Deflicker works pretty well, but if that doesn't nail it, and if I have lots of time, setting DCT=1 in the MAnalyze precursor functions that feed MDegrain can completely eliminate flicker.

Thanks for the all help.

[johnmeyer]

Originally Posted by Reel.Deel

Does it cut the video losslessly? The bitrate seems a bit too low, hence all the blocking.

Yes, Womble is a lossless MPEG-2 cutter and does not re-encode except on GOP boundaries at the head and tail of the clip. So, what I uploaded is bit-for-bit identical except at the head and tail. The low bitrate is due to encoding 125 minutes on one single-layer DVD; on setting a lower bitrate than even that would require; and on not using a very good encoder and/or not knowing how to use it. I had no control over any of that.

[raffriff42]

Thanks for the Colour Recovery links - fascinating reading.

Originally Posted by Reel.Deel

You can try V. C. Mohan's frequency domain filters (FFTQuiver and FQuiver). I don't know how well it will work since the video has lots of blocking, compromising the fixed pattern in those areas.

I did not see this post before looking into an FFT attack on this problem. I experimented with dfttest, since I already had it, skimmed the docs on creating a sigma file and made one that passes all frequencies, then setting a high value for each freq in turn looking for the magic notch filter. Well, I didn't find it - too many harmonics I'd guess. So I made one that blocks all frequencies:

8192 8192 8192 8192 8192 
8192 8192 8192 8192 8192
8192 8192 8192 8190 8192 
8192 8192 8192 8192 8192
8192 8192 8192 8192 8192
8194 8194 8192 8192 8192
8192 8198 8192 8192 8192
8192 8192 8192 8192 8192

(8192 = an arbitrary, very high noise reduction strength)

(as explained in the docs, the freq's go from 0 at top left and increase across and down - like this)


...then setting each frequency to 0 (bypass) in turn and viewing the output. If setting that frequency to 0 allowed any of the pattern through to the output, I reset it back to 8192 and moved on the the next one. (EDIT sometimes it was hard to tell if a particular freq was "good" or "bad", so a random choice was made). After going through all the freq's in this way, I had a sigma file like this:

0000 0000 8192 0000 8192 
0000 0000 8192 0000 0000
0000 8192 8194 8192 0000 
0000 8192 8192 8192 8192
8192 0000 8192 8192 8192
8192 8192 8192 0000 0000
8192 8192 8192 8192 0000
0000 0000 8192 8192 0000

And output like this:


Initial results are interesting, but not Earth shattering.
It could be optimized better if I had more patience. There needs to be a better way of tuning the frequencies.
(EDIT there is the nfile option for auto-estimating the noise spectrum, which I describe here. Not sure if it's better.)

I'm not sure I understand the usage of davidhorman's matrices - do you use them with GeneralConvolution, or mt_convolution, or something else? Here is the script I used to compare the alternatives - I used the still image, as I can't download the video ATM.

# http://avisynth.nl/index.php/Dfttest
LoadPlugin(path + "dfttest\v1.9.4\dfttest.dll")

# http://forum.doom9.org/showthread.php?t=173030
LoadPlugin(path + "variableblur07\variableblur.dll") 

ImageSource("E:\Data\Downloads\hatch pattern1.png")
ConvertToYV12

G = Last.GaussianBlur(varY=Pow(1.2, 2.0), U=-128, V=-128)
\       .Unsharp(varY=1.0, strength=1.0,  U=-128, V=-128) 
D = Last.dfttest(tbsize=1, sosize=4, sbsize=8, sfile="_misc-fft.txt")
\       .Unsharp(varY=1.0, strength=0.2,  U=-128, V=-128) 
M = Last.ConvertToRGB32
\       .GeneralConvolution(0, "
            -1  2 -1
             2  0  2
            -1  2 -1 ", 1)
\       .ConvertToYV12

return Interleave(
\    G.Subtitle("gauss")
\  , M.Subtitle("matrix")
\  , D.Subtitle("dfttest")
\ )

[johnmeyer]

Wow, that is a lot of work. Since you put that much time into it, I had to try it. I downloaded dfttest (I had everything else) and walked through my video. I like the idea of interleaving the various clips. I'll have to remember that because it lets me zoom way in (using VirtualDub's zoom levels), and still do comparisons using the entire frame. That is really important for this test, because the patterns are sometimes not easy to see unless you zoom in.

In the end, I thought the dfttest results created an artifact that looked a little like jaggies, even though it really isn't a traditional staircasing jaggie. As for convolution, while I used the convolution in Sony Vegas, the results you got from the General Convolution plugin appear to be identical to what I got from Vegas, which is what I'd expect.

So, for my purposes, I think the convolution result appears smoother and more natural. The Gaussian blur result is what I started out doing before I started this thread, and it doesn't produce results that are as pleasing because by the time you get most of the cross-hatch pattern to disappear, a lot of detail is gone as well.