Frame decimation nightmares

[infernix]

Hi all,

I am having the hardest time trying to decimate an ABC stream capture. It's 59.94 fps, it's an animation film and it does not decimate in a proper fashion with any of the decimaters I've tried.

I've tried the following:

tdecimate(mode=0,cycleR=1,cycle=2)
AssumeFrameBased().SelectOdd().tdecimate()


and all kinds of variations that I could find in the documentation. None worked.


FDecimate()


FDecimate looked like it would do the trick, but only after I had cropped the frame and disregarded the noise in the left and mostly right border. However, it didn't suffice as it still had dupes depending on the frame differences, and sectioning the entire film for threshold values is incredibly hard to do.

So I went and tried:


MultiDecimate(pass=2)


I generated the 1stpass files and tried all the various combinations with different thresholds, but none of them yielded proper frame decimation for this stream.

On top of that, I had already written a piece of avs to crop the frame properly. If you play the sample, you'll notice that the frame judders 1px up and down at completely random intervals. The only possible way to identify this would be the bottom left line, which is broken at about the 300th pixel from the left. If you watch closely you can see that it either fills a pixel line or doesnt. So I wrote a detection routine that checks whether this single line is black or is filled with video, and decide the proper crop based on that:


# s is the final output frame which we only postprocess once
s = ColorMatrix(d2v="lionking.d2v").BlindPP(quant=4,cpu2="ooooxx"). \
deblock_qed_mt2( quant1=30,quant2=45, aOff1=4,bOff1=8,aOff2=8,bOff2=12). \
hqdn3d(1,2).fft3dfilter(sigma=1). \
vmToon(ssw=3,ssh=3,strength=44,xstren=128)

# a is the frame cropped with 1 line difference to b; in essence we just crop within the juddering
# frame and crop away the line that judders at the bottom and top.
a = converttorgb(s).crop(34,1,1212,716)
b = converttorgb(s).crop(34,0,1212,716)


# here's where the crop detection routine starts. first we crop the relevant two pixel lines. the upper
# line here is always filled with video, the bottom line only is when the frame is juddering downwards;
# otherwise it is (nearly) black.
crop(34,718,464,2)

# next we add some borders to make it larger for resizing. we now have a 464x14 image.
addborders(0,0,0,12)

# use the most simple and non-smearing/aliasing resizer to blow it up to 464x140. in this image, the
# top 10 lines are the enlarged pixel line a and the next 10 lines are b. the other 120 lines are
# black and unused.
PointResize(464,140)

# change the contrast to make the difference between the two enlarged pixel lines much clearer
Levels(0, 5.3, 255, 0, 255)

# awf is the pixel line for crop A, bwf for B. we crop the rescaled 10px lines A and B and
# convert to rgb for comparison with rgbdifference.
awf = crop(0,0,464,10).ConvertToRGB()
bwf = crop(0,10,464,10).ConvertToRGB()

# here the magic happens. if the difference between line A, which is always filled with video,
# and line B, which is filled with video when we need crop A and black if we need crop B, is
# smaller than 11.5, we use crop A, else we use crop B.
c = ConditionalFilter(a, b, "RGBDifference(awf, bwf)", "<", "11.5").converttoyv12(). \
lanczos4resize(1216,720)


I might have the order wrong here, it's been a while since I wrote this avs, but you get the general idea. Anyway, this is not that relevant to the frame decimation problem, but you'll see that I've already put in a lot of effort. A sample of this code in action can be downloaded at http://dx.infernix.net/lionking/lktest.mkv and you can find the source for this sample here as well: http://dx.infernix.net/lionking/ (lionkingproblem.ts or tlk_decimate_nightmare.ts)

Right now I'm at a point where I am changing the values in mfile.txt by hand. I'm at frame 1293 of ~290000 and I don't think I have the willpower to finish it.

I've read through a lot of threads here already, and this one might be relevant: http://forum.doom9.org/showthread.php?t=122841 but it didn't help me in solving it. So my question is, is there a kind soul out there that has more knowledge of frame decimation who could look at this, please? I'm at my wit's end here. I would really appreciate it. If a longer sample is required just let me know :)

:thanks:

[Guest]

Just to play devil's advocate a bit, why does it matter with anime such as this if you get a few odd dupes now and again?

Your source material is pathological (the jumping up and down). I would try to solve that first. How did you capture this stream?

[oddball]

Because some people are perfectionists?

[Guest]

And you want us all to share your obsession? :)

[infernix]

Just to play devil's advocate a bit, why does it matter with anime such as this if you get a few odd dupes now and again?

The problems I had with decimating this stream lead to dupes that were dupes for previous frames. If it would only dupe the same frame, e.g. lets say it's 12fps and each frame is duped once, that would be acceptable. But it's duping frames before the previous one and that is really annoying to watch.

And you want us all to share your obsession? :)

Hehe, no not really. I happen to like fiddling with this and extending my knowledge, which I already did by writing that (probably crappy) piece of avisynth script, but I clearly hit a wall on the decimation here. And yes, the source is a nightmare, hence topic. Why else would I ask for help :) But I would understand if you deem this impossible. I know the capture is awful but that's simply how it is and I don't think it's re-airing any time soon for a better one.

[Guest]

OK, that sounds reasonable. I'll have a closer look at it. Maybe foxy or Xesdeeni will chip in, too.

[burfadel]

You used fdecimate(), have you tried the options with it?

You have to use the options - assuming that it is 12 fps animation:
fdecimate(fps=12,threshold=xx)

Where xx is the threshold parameter to determine which frames to decimate. The default value is 2. The threshold can be determined by doing a pass of a clip (say select few minutes, no need to compress audio) with this parameter:
fdecimate(metrics=true)
The encoded clip will have a text line up the top with a frame by frame threshold. Say if its hovering around 3 you add 3 where I wrote threshold=xx above. The value can be a floating point number, ie. 2.5

Hope this helps! It should work, fdecimate is specifically designed for non-standard decimation (as per the documentation). It actually says to use the normal decimate for 29.97-->film, ie:
telecide()
decimate()

and use it only where non standard, ie animation 12fps is required.

I take it thats correct neuron2?! I just thought i'd mention the above because I assumed he only used the fdecimate() command without options :)

[Guest]

What you say is correct, but infernix said:

"and sectioning the entire film for threshold values is incredibly hard to do"

So I assumed he was tweaking the threshold.

[infernix]

You used fdecimate(), have you tried the options with it?

I've tried with various settings. Initially I used the default, but that didn't work. Then after trying show=true and trying various thresholds I had it at 3.5 which seemed to work for my test cut. But then I ran it over the entire movie instead of the a test cut, and it was clear that the threshold could not be static like that due to the huge differences between scenes. Whatever threshold value I used, it would be fine for a few scenes but terrible for others.

So yes you're right, I didn't extensively describe how I tried fdecimate, but believe me I did. :)

[check]

well, there's always YATTA (http://ivtc.org/)

[foxyshadis]

If you want perfect, you need YATTA, no ifs ands or buts, and of course it could give a sailor seasickness, but that's not the decimation's fault. Otherwise, this does a pretty good job with only minor hiccups:

MPEG2Source("D:\tlk_decimate_nightmare_large.d2v", cpu=0)
Crop(40, 2, -40, -2)
SelectEven()
tdecimate(mode=1)

I spent a while trying to get DepanStabilize to work - obviously the recording is dying for it - in hopes that it would improve the decimation. It didn't do much for that, but it did get more watchable; unfortunately it appears to be buggy, black frames getting swapped in on sharp pans for no good reason. This is the script used:

MPEG2Source("D:\tlk_decimate_nightmare_large.d2v", cpu=0)
Crop(32, 0, -32, 0)
SelectEven()
fw=MVAnalyse(overlap=4)
d=MVDepan(last,fw,zoom=false,rot=false,range=0,info=true)
DePanStabilize(last,d,dxmax=0,dymax=1,info=false)
Crop(8, 2, -8, -2)
tdecimate(mode=1)
Maybe Fizick will test, but I need sleep.

[infernix]

I've already tried depanstabilize but it wasn't helping much. That's why I wrote the avs script that does an RGBDifference on the pixel line at the bottom left that moves up and down. As far as I can tell, this is the only possible reference to tell whether the frame is in crop state A or B. It works really well, except when the image is dark in the bottom left corner. If you watch http://dx.infernix.net/lionking/lktest.mkv you can see (bottom right) whether if it's cropped as frame state A or as B. Imagine that for all B crops, the image moves down 1 pixel, and you'll notice it's quite perfect as long as the frame is bright.

I'm experimenting with YATTA now, I'll update here when I figure it out.

[oddball]

Isn't Yatta that gay Japanese pop group? :)

[ChiDragon]

http://forum.doom9.org/showthread.php?t=126199 ... plugh's issues seem the same as yours. He also posted about this in the TDeint and TIVTC thread and I think others.

[TheRyuu]

Would this help? (http://www.kickassanime.org/phpBB2/viewtopic.php?t=3608).

I'm not an expert on this though :p