Repairing bad ivtc foreground with progressive background

[poisondeathray]

Originally Posted by StainlessS

EDIT: I trust that below format will work with whatever the RX script is

10  14
210 215

Sorry I should have posted the details. The OP couldn't post for some reason early on and we had a discussion in PM

It's just an mvtools2 interpolation function, to interpolate over bad frames. It allows for runs of more than single frames, but the problem is it requires manual identification of frame ranges. If you could ID them "automatically" with some degree of accuracy it might be a good option. It's certainly "cleaner" than QTGMC in this case, and fixes the aliasing . But we all know that's only because this example scene was "simple" - mvtools2 interpolation can make a big mess of things on more complex scenes/motion

Here is the PM

[StainlessS]

Thanx PDR, unfortunately, dont think that I'm in any fit state to do anything, at the mo.
Will come back again in the morn.

EDIT: Ive had a post from user about not being able to post, perhaps a site problem. (originally thought that I was blocking him).

[kriNon]

Hey, thought that I should post an update.

So far I've managed to run the script on the first four episodes, and it has been quite successful in finding any errors. From what I have seen the script does tend to find a lot of false positives, but it is also quite sensitive to actual errors, finding some errors so small that I couldn't even notice straight away. The script especially finds a lot of false positives when the video is either panning, shaking or fading between two clips as a transition. What I have been doing is running the script and then checking each frame output manually for errors, which has sped up this process significantly.

What I have noticed is that there are three main different types of error that are detected.
The first type being where in some frames, most visible near lines, it seems like two frames have been overlaid but with a very slight offset, resulting in double lines running parallel. This is most often not very noticeable by eye.
The second type typically occurs during motion, and is where to some limited extent, the frame has elements of the frame before/after it overlaid upon it with.
The third type is the type that is shown in the sample I posted, which results in aliasing.

It seems that each type of error would require some slightly different filtering, some individual frames might even need to be modified in an image editing program.

For now I am going to prioritise checking each episode for errors using the script, and then rechecking the list of errors manually.

I will probably end up using a mixture of QTGMC, RX, Image editing software, and possibly After Effects if I can figure out how to achieve results like what poisondeathray did.

[poisondeathray]

Originally Posted by kriNon

What I have noticed is that there are three main different types of error that are detected.
The first type being where in some frames, most visible near lines, it seems like two frames have been overlaid but with a very slight offset, resulting in double lines running parallel. This is most often not very noticeable by eye.
The second type typically occurs during motion, and is where to some limited extent, the frame has elements of the frame before/after it overlaid upon it with.
The third type is the type that is shown in the sample I posted, which results in aliasing.

Do you want to post examples of the other types ? It's not entirely clear to me but it almost sounds like you're referring to field blending ? There might be other approaches you could try

(I guess I should have mentioned this too)
In our PM's , I mentioned the NTSC version probably being better, but the OP apparently has examined all versions including broadcasted versions and apparently they all have even worse issues...

But I want to emphasize when you are combining techniques (including some manual work), you want to have access to all material that you can. It can save lots of time when you are compositing repairs. A botched part of frame on the PAL version might be able to be masked out with a NTSC version and so forth.

[kriNon]

So it turns out that I had made a mistake, and when I was looking through the list of errors I had been checking the frame numbers against the QTGMC'd track, instead of the source. Without having checked extensively, it looks like the first and second types of artifacting were caused by QTGMC and don't exist in the source. I will be rechecking errors using the proper source, and I'll post again either if I run into a problem, or when I have finished checking for errors. This might take a while.

Also, all of the issues that exist in the PAL version also exist in the NTSC version, however the NTSC version has further issues. Using the NTSC version for compositing repairs isn't an option.

[kriNon]

Alright, so I ran the script correctly this time, and after getting a list of frame numbers I then checked them manually, which resulted in me finding many damaged frames. However in the time that I was checking for errors, I found an example of an entire scene which was not detected by the script, nor was even really that close to being detected. From that I've realized that I'm going to need to find a better method of checking for errors.
Here's the scene where it wasn't detected: https://mega.nz/#!fQ8jRTKL!PR9dEmjEC...PJg5hIasyQv2UI

Nonetheless I do suspect that we found the majority of the damaged frames. I have been looking into better detection methods, however I haven't been able to find anything that is able to detect the above scene. From what I have seen I believe that the damage occurs only in the luma component of the video.

[kriNon]

So I have spent some time trying to find another better method of detection and I believe I have found one, although I do still need to figure out some of the logic. What I found is that the previous method used relied heavily on Motion Analysis to work, which meant that in some scenarios where there is little to no motion, errors will not be detected.

After doing some testing using a variety of different detection methods, I have found one that I believe will work well. What I found is that it is possible to detect the bad frames using this script:

video1=MPEG2Source("I:\[NHK]\Welcome to the NHK [DVDISO-PAL]\Disk1\episode1.d2v", cpu=0)
track2=binarize(CombMask(video1)).greyscale().invert()

ConditionalFilter(video1, video1, video1, "AverageLuma(track2)", ">", "0", show=true)
return(last)

Whenever there is an error, the AverageLuma of track2 will fall significantly, relative to the surrounding frames.

As shown in this graph here:http://i.imgur.com/5K71kII.png

It is not possible to check if the AverageLuma of track2 is below a threshold because it changes significantly throughout the episode.
I am in the process of trying to figure out a way of separating these values which do not fit the trend of the graph.
I will also need to use some kind of scene detection to run this test on a per scene basis, because the graph I posted is discontinuous at each scene change.

EDIT:
I have found a way of detecting the outliers in the data using matlab which I will post here:

%mydata is the array of AverageLuma values
x=1:length(myData);
TF=smooth(x,myData,0.1,'rloess');

Test=(TF-myData)./TF;
for i=1:length(myData)
    if Test(i)<0
        Test(i)=-1;
    elseif Test(i)
    end
end

alpha=0.15;
for i=1:length(myData)
    if Test(i)>alpha
        disp(i);
        disp(Test(i));
    end
end

From here all I would need to do is implement scene detection.

I am thinking maybe it would be best to detect scene changes, output each frame number to a text document with its corresponding scene number and AverageLuma value, and then do all of the logic in matlab.

I'll post my results tomorrow, hopefully this works!