repair frames that start & stop at wrong position [Archive]

View Full Version : repair frames that start & stop at wrong position

kaefert

11th August 2013, 15:55

Hi there!

I'm searching for a way to repair frames that don't start with the first row at the top.

for example:
http://kaefert.is-a-geek.org/misc/6TGBNMggLA/22.jpg
http://kaefert.is-a-geek.org/misc/6TGBNMggLA/23.jpg

Description of the example:
-) One frame starts with ~ row 150 at the top row
-) the bottom 150 rows are white noise
-) the next frame starts with ~ 400 rows of white noise, and ends with the top 150 rows that where missing in the frame before.

And the seperation line keeps moving slowly, so I can't hardcode it with Crop and Overlay..

raffriff42

11th August 2013, 16:45

Interesting problem - however, I don't think this is fixable. I tried to align the two parts by eye. You appear to be missing some pixels :(

http://i1226.photobucket.com/albums/ee406/XyKyWyKy/doom9t168410-merge1_zps3c86bece.jpg

kaefert

11th August 2013, 17:40

Thanks for the response!

hmm.. it seems your right, I'm missing a few rows..

But still it would be nice to have a filter that produces something like what you made out of those two sample frames, without manually stiching them together.

Do you (or anybody else reading my thread) know if there is a (half-)automated way to do this, without having to count the rows on each frame, like I would have to with Trim & Overlay?

kaefert

11th August 2013, 18:45

Okey so I've played around a little with Trim and Layer but this is a very tedious task and I really would like some automation, or maybe someone could show me an easier way?

bl = BlankClip(length=1, width=720, height=576, fps=50, color=$000000, audio_rate=48000, channels=2, pixel_type="YUY2")
sr=bf.ConvertToYUY2.ShowFrameNumber #.AssumeFPS(0.5)

rs = sr.Trim(0,14)

mis = 0 #20 #38

sp = 20

fr = sr.Trim(15,15).Overlay(bl)
fr = Layer(fr, sr.Trim(15,15).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(16,16).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

fr = sr.Trim(16,16).Overlay(bl)
fr = Layer(fr, sr.Trim(15,15).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(16,16).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

sp = 50

fr = sr.Trim(17,17).Overlay(bl)
fr = Layer(fr, sr.Trim(17,17).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(18,18).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

fr = sr.Trim(18,18).Overlay(bl)
fr = Layer(fr, sr.Trim(17,17).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(18,18).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

sp = 82

fr = sr.Trim(19,19).Overlay(bl)
fr = Layer(fr, sr.Trim(19,19).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(20,20).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

fr = sr.Trim(20,20).Overlay(bl)
fr = Layer(fr, sr.Trim(19,19).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(20,20).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

sp = 120

fr = sr.Trim(21,21).Overlay(bl)
fr = Layer(fr, sr.Trim(21,21).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(22,22).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

fr = sr.Trim(22,22).Overlay(bl)
fr = Layer(fr, sr.Trim(21,21).Crop(0,576-sp,0,0))
fr = Layer(fr, sr.Trim(22,22).Crop(0,0,0,576-sp), y=sp+mis)

rs = rs + fr

kaefert

11th August 2013, 18:49

johnmeyer

11th August 2013, 23:04

Your task seems to break down into three parts:

1. Detect the edge between real video and noise, and save the two scan line numbers where those two lines of demarcation happen.

2. Shift & crop one portion of the video up and the other down so that they line up with each other. Add an offset equal to the number of missing scan lines.

3. Fill in the missing lines with duplicates of the lines above and below, perhaps using some blur or other function to make the line less obvious.

You didn't post a video, so I don't know whether the location of the line of demarcation changes, but if it does, then you will find lots of help in this thread from a few months ago where StainlessS helped me by providing some amazing functions that let you detect edges:

Bad 1950s Kinescope - Hopeless? (http://forum.doom9.org/showthread.php?t=167591)

That will help you with #1 above.

#2 above is just basic AVISynth "blocking and tackling" where you simply shift and crop by the numbers provided by #1.

Once you've conquered #2, then #3 is actually almost the same, with the addition of a blur.

So, take a look at the linked thread and see if it provides some help with the detection portion of your problem.

kaefert

11th August 2013, 23:08

So this does the same as the copy-paste pattern script posted before, but with the next 2 frames I've hit a wall with that function, they need to be cut and stiched differently..

function restoreSplitFrame(clip twoframes, int split, int miss)
{
f1 = twoframes.Trim(0,-1)
f2 = twoframes.Trim(1,-1)

blf = BlankClip(length=1, width=720, height=576, fps=50, color=$000000, audio_rate=48000, channels=2, pixel_type="YUY2")

l1 = Layer(f1.Overlay(blf), f1.Crop(0,576-split,0,0))
l1 = Layer(l1, f2.Crop(0,0,0,576-split), y=split+miss)

l2 = Layer(f2.Overlay(blf), f1.Crop(0,576-split,0,0))
l2 = Layer(l2, f2.Crop(0,0,0,576-split), y=split+miss)

return l1 + l2
}

sr=bf.ConvertToYUY2.ShowFrameNumber #.AssumeFPS(0.5)
rs = sr.Trim(0,14)

mis = 0 #20 #38

rs = rs + restoreSplitFrame(sr.Trim(15,16),20,mis)
rs = rs + restoreSplitFrame(sr.Trim(17,18),50,mis)
rs = rs + restoreSplitFrame(sr.Trim(19,20),82,mis)
rs = rs + restoreSplitFrame(sr.Trim(21,22),116,mis)
rs = rs + restoreSplitFrame(sr.Trim(23,24),150,mis)
rs = rs + restoreSplitFrame(sr.Trim(25,26),230,mis)
rs = rs + restoreSplitFrame(sr.Trim(27,28),340,mis)

But I guess my code should show what I would like to have. Some way to detect the switch automatically and crop and stich those half pictures together automatically, without me having to determine in which row of my picture the "true" first row starts..

edit: @johnmeyer thanks for the post (didn't see it when I posted this), yes that sounds reasonable, I'm gonna look at that link tomorrow, girlfriend wants to go sleep now.

kaefert

12th August 2013, 09:46

Okey, so I did take a look at that thread, and this detection stuff with masks and such looks really complacted.

also the automation is not as simple as we hoped, since the scheme doesn't stay the same for long.

Here's the relevant portion of my script, and I think I'm gonna leave it at that:

function restoreSplitFrame(clip twoframes, int split, int space, int "select", bool "reverse")
{
select = default(select, 2)
reverse = default(reverse, false)

f0 = twoframes.Trim(0,-1)
f1 = twoframes.Trim(1,-1)

reverse ? Eval("""
up = f1
dwn = f0
""") : Eval("""
up = f0
dwn = f1
""")

blf = BlankClip(length=1, width=720, height=576, fps=50, color=$000000, audio_rate=48000, channels=2, pixel_type="YUY2")

return (select == 0) ? Eval("""
f0.Overlay(blf).Layer(up.Crop(0,576-split,0,0)).Layer(dwn.Crop(0,0,0,576-split), y=split+space)
""") : (select == 1) ? Eval("""
f1.Overlay(blf).Layer(up.Crop(0,576-split,0,0)).Layer(dwn.Crop(0,0,0,576-split), y=split+space)
""") : Eval("""
l1 = f0.Overlay(blf).Layer(up.Crop(0,576-split,0,0)).Layer(dwn.Crop(0,0,0,576-split), y=split+space)
l2 = f1.Overlay(blf).Layer(up.Crop(0,576-split,0,0)).Layer(dwn.Crop(0,0,0,576-split), y=split+space)
return l1+l2
""")
}

#return bf.ShowFrameNumber.AssumeFPS(5)
sr=bxd.ConvertToYUY2 #.ShowFrameNumber #.AssumeFPS(0.5)

mis = 2 #0 #20 #38

rs = sr.Trim(0,14)
rs = rs + restoreSplitFrame(sr.Trim(15,16),20,mis)
rs = rs + restoreSplitFrame(sr.Trim(17,18),50,mis)
rs = rs + restoreSplitFrame(sr.Trim(19,20),82,mis)
rs = rs + restoreSplitFrame(sr.Trim(21,22),116,mis)
rs = rs + restoreSplitFrame(sr.Trim(23,24),150,mis)
rs = rs + restoreSplitFrame(sr.Trim(25,26),230,mis)
rs = rs + restoreSplitFrame(sr.Trim(27,28),340,mis)
rs = rs + restoreSplitFrame(sr.Trim(27,27)+sr.Trim(29,29),340,mis,1)
rs = rs + restoreSplitFrame(sr.Trim(30,31),440,mis)
rs = rs + restoreSplitFrame(sr.Trim(31,32),500,mis,1,true)
rs = rs + sr.Trim(33,35)
rs = rs + restoreSplitFrame(sr.Trim(36,37),540,mis)
rs = rs + sr.Trim(38,50)
rs = rs + restoreSplitFrame(sr.Trim(51,52),200,mis,0,true)
rs = rs + restoreSplitFrame(sr.Trim(52,53),200,mis)
rs = rs + restoreSplitFrame(sr.Trim(54,55),400,mis)
rs = rs + sr.Trim(56,5490)

blf = BlankClip(length=1, width=720, height=576, fps=50, color=$000000, audio_rate=48000, channels=2, pixel_type="YUY2")
rs = rs + sr.Trim(5491,5491).Layer(blf, y=38)

StainlessS

12th August 2013, 15:04

A video sample would not be amiss.

kaefert

12th August 2013, 15:16

A video sample would not be amiss.

hmm, can you point me to an avi-encoder that I can run from the windows (wine) command line that will understand avs files as input?

With a lot of playing around I've managed to get the x264 tmod windows build to produce me nice mp4 files (with working sound ;) ) that seem to be able to play everywhere I want them to, but I don't think x264 encoded mp4 files are a good format as input for avisynth, at least not when you want to be frame accurate, which I think will be necessary for this problem.

edit: well, for the meantime, here's an h264 mp4 (9,4MB):
http://kaefert.is-a-geek.org/misc/6TGBNMggLA/first125frames_25i.mp4

and here the deinterlaced clip (9,1MB):
http://kaefert.is-a-geek.org/misc/6TGBNMggLA/first125frames_QTGMC-250_50p.mp4

StainlessS

12th August 2013, 15:31

Cant you run VirtualDub under Wine ?

I think there is something called AvstoAVI (or AVS2AVI) which I believe is command line (no idea how or how well it works).
another crazy solution, is linked at end of ths thread:http://forum.doom9.org/showthread.php?t=167558

kaefert

12th August 2013, 15:55

I can run VirtualDub under wine, but it doesn't want to open my avs files, which work perfectly when opening them with mplayer, the x264 encoder, or the avs2yuv tool that allows me to pass the video stream over to linux utilties, but without sound..

The error VirtualDubMod prints me: "AVI Import Filter error: file open error (8004406f)"

Since I've searched for something like this already I've already stumbled upon AVS2AVI yesterday evening, which has a dead link from its forum post here, but is still findable through google.
I've found this version here:
http://kaefert.is-a-geek.org/misc/6TGBNMggLA/AVS2AVI.zip

This gives the error:
AVIFileOpen failed, unable to open "Z:\path\name.avs" : Ung³ltiges Handle.

fixme:avifile:AVIFileExit (): stub!

I'm gonna take a look at the "crazy solution" you linked now

StainlessS

12th August 2013, 16:24

I've converted the mp4's to AVI (ffmpeg).
Have to say, I admire your optimism, not sure I can assist at all here. :(

kaefert

12th August 2013, 16:33

I've converted the mp4's to AVI (ffmpeg).
Have to say, I admire your optimism, not sure I can assist at all here. :(

okey, well its alright, I've repaired a few frames with the script I posted in the morning, and I've replaced the complete white noise frames with the BadFrames plugin (also shown in my script above) and I think I'm gonna leave it like that - I think its watchable without throwing up.

kaefert

12th August 2013, 16:38

And I didn't get the TWriteAvi Filter working... It wil give me a gui window to select the codec, but depending on what codec I choose, I get either no file at all, or I get an avi file that contains exactly 16 777 216 = 0x1000000 bytes but everything filled with zeros

StainlessS

12th August 2013, 17:04

Just checked, TWriteAVI uses source from an old version of VDUB, I guess as VDUB dont work, then TWriteAVI could well
suffer from same problems. I would guess that others have in past posted on getting VDub working under Wine, might be
worth a search.

from a Vapoursynth thread:

Originally Posted by IsoaSFlus View Post
hello Myrsloik,how can vapoursynth preview the video frame by frame in linux?
May i suggest wine and virtualdub?

I guess if anybody knows, Myrsloik would and so it must be possible.

Plenty of hits searching "VirtualDub Wine". (EDIT: 5 pages).

kaefert

12th August 2013, 17:36

Plenty of hits searching "VirtualDub Wine". (EDIT: 5 pages).

Okey, I'm gonna take a look at that. I didn't have to much interest in getting VirtualDub to work until now, I've got my workflow already worked out, And I really love that about Avisynth, that I don't need to use a GUI :)

creaothceann

12th August 2013, 17:53

You could always use VirtualDub to install Windows, and give it access to your files via shared folders (read-only if you insist).

StainlessS

12th August 2013, 18:03

OK, maybe I'm a total pessimist, perhaps this can assist.

AVISource("D:\AVS\AVI\first125frames_25i.mp4.AVI")
assumebff()
ConvertToYUY2()
ORG=Last
Z=ORG.Zebra(cmad=9,lo=0,hi=47)
Z=Z.ScriptClip("""
Bingo=RT_YInRangeLocate(baffle=8,x=5*48+48/2-2,w=4,lo=174,hi=255)
Last=(Bingo) ? Last.Layer(Last.BlankClip(width=YIRL_W,height=YIRL_H,color=$FF0000),x=YIRL_X,Y=YIRL_Y) : Last
Return Last
""")

return stackHorizontal(ORG,Z)

Locates black bars in your clip, detected bars hilited in RED, might need to play with settings to get optimal.

here result:

https://s20.postimg.cc/obillq7b1/Test_zps2cb9e0b8.jpg (https://postimg.cc/image/qsucszr7d/)

EDIT: Need RT_Stats and Zebra, see MediaFire in sig.

PS, Dont believe a word John Meyer says, he did all the work in the other thread, I just wrote a couple of simple plugins.
EDIT: Also, Post #10 in John's thread by IanB, contains goodies, dont overlook.

EDITED numbers in script

kaefert

12th August 2013, 18:06

You could always use VirtualDub to install Windows, and give it access to your files via shared folders (read-only if you insist).

I would need to setup a windows pc, whch I would prefer not to.

My VirtualDub Error code seems to be related to permission problems, I've found the same error code here:
http://www.techsupportforum.com/forums/f217/cant-play-certain-files-in-other-user-accounts-322598.html

I've tried quite a few guides about virtualdub in wine but nothing changed in its behaviour..

@StainlessS thanks for your time! that looks promising. Can white noise also be detected like that?

wonkey_monkey

12th August 2013, 18:14

I would need to setup a windows pc, whch I would prefer not to.

I think creaothceann may have meant VirtualBox, not VirtualDub :)

StainlessS

12th August 2013, 18:21

The Zebra Bars are for YPlane,

1 ) YPlanemin (0->255)
2 ) YPlanemax (0->255)
3 ) YPlaneminmaxDifference (0->255)
4 ) YPlaneMedian (0->255)
5 ) AverageLuma (rounded to nearest int, range 0 -> 255)
6 ) YInRange (population of pixels in range lo to hi, scaled 0->255 and rounded to nearest int)

I used YInRange band

Zebra(cmad=9,lo=0,hi=47)

The greater the percentage of pixels in range 0 to 47, the nearer white, the bar. (sometimes other bars may be more appropriate),
Try Zebra alongside your clips to see which if any provide a reasonable detector, and then use RT_YinRangeLocate() which
'looks for' a block of the band in a particular range.
EDIT: If darkest black were eg 16 then lo should perhaps be set likewise. 47 allows for lighter noise (for Zebra YInRange band).

Can white noise also be detected like that?

Let me know. (dont see why not, EDIT: guess it depends on what you mean by white noise, John Meyer's problem was light bands
rolling upwards).

EDIT: Each scanline of the bars represents the contents of the corresponding scanline of the clip,
best to show the Zebra bars alongside the clip and which (if any) bar best matches the faults you
are trying to locate. Zebra also does vertical 'scanline' bars which can be stacked eg below clip.

EDITED numbers from script.

kaefert

12th August 2013, 18:32

I think creaothceann may have meant VirtualBox, not VirtualDub :)

well thats a performance killer. (in my experience)

Let me know. (dont see why not).

Okey, I'm gonna try it, and post back when I have results.

creaothceann

12th August 2013, 19:36

I think creaothceann may have meant VirtualBox, not VirtualDub :)

Yes. :o

johnmeyer

12th August 2013, 23:11

PS, Dont believe a word John Meyer says, he did all the work in the other thread, I just wrote a couple of simple plugins.Well, it's true I didn't just copy/paste some code, but I would have been unable to proceed without those "simple plugins." They actually are amazingly useful for getting your script to make decisions based on visual problems that are obvious to the naked eye, but difficult sometimes to pinpoint programatically. That is one of the main problems that needs to be solved here which is why I recommended reading that thread.

StainlessS

13th August 2013, 01:36

I also have to say that RT_YInRange() was totally down to Martin53, his idea (and a real good one),
I thought the RT_YInRangeLocate() might add something to his idea, but without his initial idea,
there would be nothing to build upon. Nice one M53.

EDIT: M53, I shall read your PM of several weeks ago again, & see what I can do.
[I might require clarification, (or something), but consider it coming.]

raffriff42

13th August 2013, 02:39

My non-Avisynth-wizard method:
repair_720x576-pass7.mp4 (https://dl.dropboxusercontent.com/u/108089426/Video_temp/repair_720x576-pass7.mp4) (dropbox - 2.5MB) Export as image sequence;
Copy all relatively undamaged frames to new folder (60 out of 250 in this case)
Import as new image sequence;
Deshaker to fix those frames with only minor jumping / noise bars;
Overlay on existing video in Premiere for lining-up purposes;
Time stretch to fake all those missing frames;
Slow zoom & pan to roughly match existing final few frames.
A few brightness tweaks to lessen the most jarring changesIt's a little rough - no, it's very rough :D