Noob question: What massage would allow shift vs brightness comparison?

[MrPete]

I'm not exactly sure how to even describe this. Below are three frame-extracts. They are not going to line up perfectly (partly because they are actually manual screen grabs as should be obvious).

The actual difference:
Cap 1: Baseline
Cap 2: Cap 1 but with pulldown; also *slightly* distorted
Cap 3: Next frame, clearly shifted a little everywhere

What I want: a way to transmogrify the data so I can compare 1-2 and see that they are "more similar" than 1-3, because 1-2 has less shift.

I *think* that means I want to totally discount the fact that #2 happens to be: blurrier, lighter, etc.

So... what kind of transformation on the data could help accomplish this? AND, how do you think about such a question?

I'm very much in the early stages of learning how to think about these things properly, so probably as much as if not more than "the answer", I am very interested in how you would approach such a question!

Thanks sooo much, and Happy New Year!
pete

Cap1:
Cap2:
Cap3:

[johnmeyer]

If these are frames in a video, and if your main objective is to make the frames align, using the sprocket holes as alignment guides, you could use Deshaker inside of Virtualdub, and restrict deshaker so it only looks at pixels from the left side of the frame. It will then adjust all frames so the sprocket holes are in exactly the same X-Y location, from frame-to-frame.

[MrPete]

I do use Deshaker, but that comes after I have rejected the frames that I don't want.

My goal for this phase is to discover which sets of frames correspond to a single film frame.

In my example, the first two captures are for one film frame, while the third capture is for the next frame.

I've tried pixel-difference counting (RT_LumaPixelsDifferent) and luma-difference (RT_YDifference). Neither one provides useful results... because the luma difference for the first two frames is so large.

Thinking "out loud"... I've seen edge-detection algorithms that reduce a photo to a sharpened outline of edges. Is it possible that this can be done in AVIsynth? If so, perhaps that would help me deted edge-motion...

[johnmeyer]

I designed a film to transfer system ten years ago. I removed the shutter from the projector, run it at full speed, and point the camera directly into the projector lens at point-blank range. I then use software to detect and remove fields that either contain pulldown (where the film is moving). These fields do not match either adjacent field. Once that is done, I then decimate any fields that are duplicates.

I would suggest that you look at TIVTC which includes the field matching software (TFM) and the field decimation software (TDecimate). You may also find a use for SRestore, even though it was built for another purpose.

[MrPete]

Thanks, John. I'll take a look! You're correct: it's the recognition of pulldown fields that is challenging me right now.

[MrPete]

One tiny problem: TFM appears to strongly assume interlaced video. Mine is not. It is NEVER combed.

Do you suspect it might be able to field-match even with no combs at all?

[johnmeyer]

Non-interlaced (progressive) video is the same as interlaced video, but with no temporal difference between "fields" (although the field concept obviously doesn't apply). Most plugins/filters designed for interlaced will work for progressive. The reverse, of course, is not true at all. "Combs" are actually a residual artifact in some telecine, but not all. They are not part of the actual matching and decimation process, but instead are usually handled by post processing (see the "pp" parameter in TDecimate).

[MrPete]

That's all true, John.

However (based on code examination and now completing a few experiments), the TFM() routine depends entirely on discovering combing to detect which fields/frames to be removed

It's in the code, it's in the intermediate file (if two-pass is used)...

My result: same input and output every time

This is essentially what I saw with the RestoreSuper8Frames() routine. It too is searching for combing to discover frames to be removed.

That's why I have been going down this alternate path...

[johnmeyer]

Why don't you post some sample video so we can actually see what's going on?

[MrPete]

Sure! Here's some fresh samples online:

Here is the shortest subclip -- just 26 frames.
Same clip, ~100 frames (80MB)
More. Over 200 frames (223MB)

These are YV12, encoded using Lagarith lossless to avoid losing anything... http://lags.leetcode.net/codec.html

[johnmeyer]

Sorry, I can't play the video. Someone else will have to help.

[feisty2]

Originally Posted by johnmeyer

Sorry, I can't play the video. Someone else will have to help.

I loaded the video by lwlibavvideosource in avisynth and it returns a 1080p yv12 clip with no error

[MrPete]

I use lagarith lossless compression to retain all bits. Is that bad?

If so... what's the usual thing for online sharing here?

Sorry to be such a noob

[feisty2]

you might wanna try x264 -- qp 0 for lossless encoding, it's a little bit slower and the file size should be also a little bit smaller, and pick ".mp4" as the container
the point is such combo (h264 + mp4) could be played everywhere, like USD in currency

[MrPete]

@feisty -- I **like** that idea.

Can you point me to a tutorial on how to use it effectively?

I loaded it up... used those options... and got a 250% increase in size and an mp4 that doesn't play for me

Obviously I'm doing something wrong...

[feisty2]

x264.exe --output video.mp4 --preset ultrafast --qp 0 input.avs

[MrPete]

Wow.

I feel like I just ran into a political buzz saw. There's apparently no official x264 build with mp4 support? I'm honestly not sure why; it doesn't seem worth reading through all the back-and-forth on this.

If you know of a place for downloading x264.exe WITH mp4 support, I'd love to hear about it. Otherwise, I can't imagine why any normal person would try to use x264 for creating mp4 files (one of its primary purposes) if it can't actually create mp4 files.

I don't want to get into the details of x264 vs mp4 and why/why-not here. I just want a solution.

(If there's something valuable I can learn or contribute by getting into it, then please aim me in the right direction... but this feels like a huge distraction!)

[feisty2]

uh... I didn't know the official x264 builds have no "mp4" support, sorry for that
I use x264 tmod for everyday encoding, and it supports "mp4" output,
you can get it here
binary

[creaothceann]

MKV > MP4 anyway...

[feisty2]

Originally Posted by creaothceann

MKV > MP4 anyway...

however it works inside really doesn't matter (at least to me), it gives you the wanted mp4 file and without an extra manual mkv to mp4 remux, that's good enough to me