Repairing bad ivtc foreground with progressive background

[kriNon]

I have a progressive DVD where every so often I will find a section where the foreground has been IVTC'd incorrectly and then put over a progressive background.

Is there any way I could repair the IVTC on the foreground while leaving the background still mostly unaffected?

I have found some success with using QTGMC(InputType=2), however this resulted in some issues including splodgy noise, haloing, detail loss, in other parts of the video.


EDIT: New sample: https://mega.nz/#!LQEWmIrS!dqF4mN8YB...cGgM1F0H3VjaQ8

[poisondeathray]

Did you try something simpler / better first ? Like custom IVTC settings ? Likely your PP settings are causing those issues, it's deinterlacing those frames when it shouldn't be

[kriNon]

Yeah, so I have tried using multiple different deinterlacing and IVTC tools and so far the only thing to have any real success is QTGMC. I have spent a lot of time tweaking QTGMC, but honestly I don't get much better of a result than just using QTGMC(InputType=2). Are you saying I should try a different SrchClipPP?

[poisondeathray]

Post a sample of original source

[kriNon]

the sample I posted was mostly unprocessed, it was just resized. Here is a completely unprocessed sample though: https://mega.nz/#!fB9ggQzK!K3wEkK--d...BvItO-L5DtBJ2Q

[kriNon]

The sample I posted earlier was just resized with no further processing, here is a completely unprocessed sample though:
https://mega.nz/#!fB9ggQzK!K3wEkK--d...BvItO-L5DtBJ2Q

[kriNon]

What I linked was pretty much the original source, but it had been resized. Here is a sample that hasn't been resized though in case that helps: https://mega.nz/#!fB9ggQzK!K3wEkK--d...BvItO-L5DtBJ2Q

[kriNon]

Here's a copy of the original source: https://mega.nz/#!LQEWmIrS!dqF4mN8YB...cGgM1F0H3VjaQ8

I was thinking that it would be good to be able to detect which frames have issues, and then I could either edit them manually or use QTGMC on only these frames.

I was thinking one way of detecting these frames could be to QTGMC the entire source, and then look at the difference between the QTGMC'd video track and the source. If they are different enough I could potentially also check for aliasing to detect these frames more accurately.

Although I would like to hear if anyone has any better ideas about how to implement this?

[LemMotlow]

Originally Posted by kriNon

Here's a copy of the original source: https://mega.nz/#!LQEWmIrS!dqF4mN8YB...cGgM1F0H3VjaQ8

Looks to me like your source is plain progressive frames encoded as interlaced, having been badly deinterlaced or inverse telecined earlier. The only thing you'll get by deinterlacing is duplicate frames.

[kriNon]

Originally Posted by LemMotlow

Looks to me like your source is plain progressive frames encoded as interlaced, having been badly deinterlaced or inverse telecined earlier. The only thing you'll get by deinterlacing is duplicate frames.

Yeah, that's exactly what has happened to the source. My main goal right now is to try to find a way of detecting these frames where it was badly deinterlaced/ivtc'd. From there I can either try to find a way of fixing it, possibly by interpolating the bad frames, manually editing the frames or finding a filter that works (possibly QTGMC) and only running it on frames where the issue is detected.

Also, although you wouldn't expect a deinterlacing to do anything to help with this issue, QTGMC actually works fairly well. The issue is that it causes some issues in other parts of the video, such as enhancing noise and creating haloing.

[kriNon]

Originally Posted by LemMotlow

Looks to me like your source is plain progressive frames encoded as interlaced, having been badly deinterlaced or inverse telecined earlier. The only thing you'll get by deinterlacing is duplicate frames.

Yeah, that's exactly what's happening with the source.

My current aim is to find a method of detecting which frames have this issue, after which I can either manually edit these frames, or filter just these frames.

Although unexpected, unlike most deinterlacers QTGMC actually works fairly well for fixing these frames. The issue is that QTGMC creates noise and introduces haloing. As such if I am to continue using QTGMC, I would like to limit it to frames that actually need it, hence the need for detecting which frames need to be processed.

Alternatively I could edit these frames manually once I have found all the frames that require editing.

[StainlessS]

ClipClop() can replace frames/ranges of a clip with same frames from one of up to 255 replacement clips.:- http://forum.doom9.org/showthread.ph...light=clipclop

An editing utility SawBones/FrameSurgeon, allows use of VirtualDub to view eg multiple clips in a 2x2 stacked window, and mark for replacement frames/ranges in src with same range from another clip, can also eg delete frames/ranges. Framesurgeon uses ClipClop and Prune plugins for replacement and deletion.

Sawbones/FrameSurgeon:- https://forum.doom9.org/showthread.p...light=sawbones

Prune: https://forum.doom9.org/showthread.p...ighlight=prune

So, could do main processing now, and later replace dodgy frames when you figure out how.

If an auto detection script is written, can just create a frames file for use with ClipClop.

EDIT: EG,

1 100,-10  # replace 10 frames, 100 -> 109 with frames from replacement clip 1
1 100,109  # same
1 100 109  # same
1 100        # just frame 100, with rep clip 1

[kriNon]

I will definitely look into using ClipClop with QTGMC to replace any bad frames. Sawbones/FrameSurgeon look useful in a worst-case scenario, however I would much prefer not to have to look through 24 episodes manually looking for issues.

I would prefer to do my main processing later as I am still finalising some of my filtering and I'd like to get the encode all done at once.

Not sure how I should go about doing an auto-detection script though, if anyone has any ideas please let me know. I'm probably gonna give it a go tomorrow or the next day.

Ideally I'd like to find a faster/more efficient method to running the entire video through QTGMC and then looking for frames with a large enough difference to the original.

[StainlessS]

However you do it, your gonna wanna do the QTGMC thing first, as you will need access to that result to use any kind of auto
frames file generator (otherwise you would need to use something other than ClipClop to replace frames, it needs the frames file
before it starts.) You likely would also want to view frames to be replaced, unless QTGMC output already existed, its gonna take
an awful long time (probably multiple view times) for you to do anything.

EDIT: I thought you were in NTSC land (just looked at your sample).
That problem is quite horrible, and difficult to detect when its happening I would think.
QTGMC has not got much chance to improve that, was horrible to begin with (when playing in DGIndex).

d2v file

DGIndexProjectFile16
1
D:\New Folder\VTS_02_1.demuxed.m2v

Stream_Type=0
MPEG_Type=2
iDCT_Algorithm=6
YUVRGB_Scale=1
Luminance_Filter=0,0
Clipping=0,0,0,0
Aspect_Ratio=16:9
Picture_Size=720x576
Field_Operation=0
Frame_Rate=25000 (25/1)
Location=0,0,0,1368

900 5 0 0 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 538035 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1077074 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1615337 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 2154563 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 2693421 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 3232202 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 3772038 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 4310295 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 4848535 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 5386604 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 5924738 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 6462888 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 7000902 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 7538770 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 8075756 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 8612069 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 9150842 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 9688120 0 0 0 32 32 92 b2 b2 a2 b2 b2 a2 b2 a2 ff

FINISHED  100.00% VIDEO

UpConv=1 made no difference.

VideoFileName   ="D:\New Folder\out.d2v"
MPEG2Source(VideoFileName,UpConv=1)      # Upconv to convert to YUY2, unmangle chroma if encoded as interlaced.
#TFM(d2v=VideoFileName) TDecimate(mode=(TFM_ANIME)?1:0)
Return Last

EDIT: And the log file (Shows Interlaced all the way through playing in DGIndex, up until last frame where displays Progressive).

Stream Type: Elementary
Profile: main@main
Frame Size: 720x576
Display Size: [not specified]
Aspect Ratio: 16:9 [3]
Frame Rate: 25.000000 fps
Video Type: PAL
Frame Type: Progressive # EDIT: I think this just shows whatever it showed in DGIndex on last frame
Coding Type: B
Colorimetry: BT.470-2 B,G*
Frame Structure: Frame
Field Order: Top
Coded Number: 227
Playback Number: 227
Frame Repeats: 0
Field Repeats: 0
VOB ID: 
Cell ID: 
Bitrate: 8.875 Mbps
Bitrate (Avg): 8.975 Mbps
Bitrate (Max): 9.079 Mbps
Timestamp: 
Elapsed: 0:00:09
Remain: FINISH
FPS: 24.92
Info:

[kuchikirukia]

If QTGMC is giving you haloing try turning down the sharpness.

[kriNon]

So I believe I have found a way to detect the bad frames.

Using this script:

SetMemoryMax(2048)
video1=MPEG2Source("I:\[NHK]\Welcome to the NHK [DVDISO-PAL]\Disk1\episode1.d2v")
video2=video1.QTGMC(InputType=2)
video3=subtract(video1,video2)
video4=subtract(BlankClip(video3),BlankClip(video3))
compare(video4, video3)

compare will give an output at the top of the screen, and from what I have seen, if Mean Abs Dev is greater than about 0.97 then it will be a bad frame.

Does anyone know how I could get the value of Mean Abs Dev from the compare function, and then if it is greater than 0.97 output the frame number, ideally into a text file?

I guess I'll probably need to use a combination of ConditionalFilter and WriteFile? How would I go about getting an output from compare though? Any ideas?

[poisondeathray]

Originally Posted by kriNon

So I believe I have found a way to detect the bad frames.

Using this script:

SetMemoryMax(2048)
video1=MPEG2Source("I:\[NHK]\Welcome to the NHK [DVDISO-PAL]\Disk1\episode1.d2v")
video2=video1.QTGMC(InputType=2)
video3=subtract(video1,video2)
video4=subtract(BlankClip(video3),BlankClip(video3))
compare(video4, video3)

compare will give an output at the top of the screen, and from what I have seen, if Mean Abs Dev is greater than about 0.97 then it will be a bad frame.

Does anyone know how I could get the value of Mean Abs Dev from the compare function, and then if it is greater than 0.97 output the frame number, ideally into a text file?

I guess I'll probably need to use a combination of ConditionalFilter and WriteFile? How would I go about getting an output from compare though? Any ideas?

I don't think compare() (PSNR) has an equivalent internal runtime function . But you might be able to use something similar like lumadifference

This seems to work ok on your test clip with a threshold of 1.31 (use show=false to disable the debug overlay) . You might have to tweak the value for other sections

video1=MPEG2Source("VTS_02_1.demuxed.d2v", cpu=0)
video2=video1.QTGMC(InputType=2)
video3=subtract(video1,video2)
video4=subtract(BlankClip(video3),BlankClip(video3))

ConditionalFilter(video4, video2, video1, "LumaDifference(video4,video3)", ">", "1.31", show=true)

[kriNon]

Yeah, that works perfectly poisondeathray, thanks!

I'll have a look at figuring out how to output the frames to a text file using WriteFile tomorrow. Looks like detection works and is fairly consistent.

[poisondeathray]

Originally Posted by kriNon

Yeah, that works perfectly poisondeathray, thanks!

I'll have a look at figuring out how to output the frames to a text file using WriteFile tomorrow. Looks like detection works and is fairly consistent.

WriteFileIF() takes the same basic syntax

video1=MPEG2Source("VTS_02_1.demuxed.d2v", cpu=0)
video2=video1.QTGMC(InputType=2)
video3=subtract(video1,video2)
video4=subtract(BlankClip(video3),BlankClip(video3))



WriteFileIf(video4, "log.txt", "LumaDifference(video4,video3) > 1.31", "current_frame")

But the log will write the current frame number when value >1.31 (or whatever you specify as the condition) . When you have strings of consecutive frames detected e.g. 206,207,208,209, they will be on separate lines . So if you were to use the RX() interpolation function as discussed in PM, you would need to group them somehow

[StainlessS]

Originally Posted by StainlessS

and difficult to detect when its happening I would think

Am delighted to be proved wrong

you would need to group them somehow

From FrameSel:- https://forum.doom9.org/showthread.p...light=framesel

FrameSel_CmdReWrite(Clip c,String ofn, int F1, ... , int FN, String "SCmd"="", String "Cmd"="", \
      Bool "Reject"=False, Bool "Ordered"=True, Bool "Debug"=False, Bool "Range"=True, Bool "Space"=False, Bool "Prune"=False)

Args identical to FrameSel (up to Debug), with exception of the additional compulsory 2nd String arg which is  
an output filename, and missing Ver and Show and Extract args.
See FrameSel() description for other arguments.

      ofn,      String, Compulsory.
                Output file name. It is advised to use a different name to the Cmd file, although there is nothing to stop you
                overwriting an original Cmd file if you like a little gamble.(Output file only written after fully read in and
                all Ordering, Rejecting etc already processed. Only likely error would be during file writing eg eject floppy disk).

      Range,    bool, Default=True
                False = Write individual frame numbers only, to output file.
                True  = (Default) Write comma separated Frame Ranges to output file (only where adjacent frames).

      Space,    bool, Default=False
                False = (Default) Write comma separators in output file for ranges.
                True  = Write SPACE separators in output file  instead of comma for ranges.
      
      Prune,    bool, Default=False
                False = (Default) Write FrameSel style frames or ranges to output file.
                True  = Write Prune style frames or ranges to output file (with a clip index first eg '0,10,20' instead of '10,20').
                Prune supports up to 256 input clips that is the reason it needs a clip index.               
                Prune Supports Audio (whereas FrameSel does not) and also allows you to audio fadein/out at splices.
                This function can only output Prune command files, it cannot read Prune command files.  

FrameSel_CmdReWrite, takes either Frame numbers as direct arguments, or in the SCmd string, or in the Cmd file (at least one must
be specified). If more than one command method is used, will process Cmd file first, then SCmd string second and lastly
directly supplied frame numbers. The command frames will be acted upon via the Reject, and Ordered, args (as in FrameSel)
and then re-written to the output command file. Any comments in SCmd string or Cmd file will not be written to the output
command file.

The FrameSel_CmdReWrite() function returns the total number of frames written to the output file (including ranges).

This function could have several uses:-
  To combine multiple frame specifying command methods into a single command file, and if Ordered, then remove duplicates. 
  Convert single frame command files to frame range command files, (smaller and perhaps easier to peruse).
  Convert frame range command files to single frame command files, for use in another plugin that expects only frame numbers.
  Convert a Reject=True command set to Reject=False command file, and vice versa.
  Convert a command file of single frames/ranges to reject from a clip, into a Prune command file (Prune does not itself have a
    Reject arg but it does fadein/out audio at splices and can also coalesce single frames into ranges before splicing so that
    it does not fadein/out at every single individually specified frame).
  Perhaps more uses.
  
  *** NOTE ***, if you would normally use Reject=True in FrameSel, and you do a conversion with Reject=True, then the resultant
  command file should be used with Reject=False in FrameSel.

Something like:- (In separate script after creating frames file).

c = Avisource(...) # Needs source clip, only for Reject mode really (so it knows how long the clip is and what to reject),
SPACE =True # Space separator, else Comma
FrameSel_CmdReWrite(c,"OutputRanges.txt",Cmd="InputFrames.txt",Range=True,Space=SPACE)
Return MessageClip("Done")

EDIT: I trust that below format will work with whatever the RX script is

10  14
210 215

or SPACE=False

10,14
210,215