Experimental Fieldblend Reversal/Removal (prev. "RestoreFPS")

[Mug Funky]

kenshin? that's originally on film, so it's probably an oldschool telecine with peculiar blending. that's my guess, anyway. there's a LOT of kenshin stuff out there...

@backwoods: there's a good NTSC master of Violent Cop here... too bad it got put throught the blender for release in PAL :). i don't have time to encode it as NTSC with auto pulldown detection...

[edit]

come to think of it, hong kong is both NTSC and PAL, so it's possible they sent a PAL master to the US and an NTSC master to Australia :o

[Backwoods]

@backwoods: there's a good NTSC master of Violent Cop here... too bad it got put throught the blender for release in PAL :). i don't have time to encode it as NTSC with auto pulldown detection...

[edit]

come to think of it, hong kong is both NTSC and PAL, so it's possible they sent a PAL master to the US and an NTSC master to Australia :o

Whatever they did, it's borked majorly. I have a buddy in the UK who has his eyes peeled for that release (white cover one, correct?). Hopefully he picks it up soon.

MOmonster's method posted in the old thread concerning this material works...to an extent. It makes the movie watchable. I'll post the results in that thread soon.

/derail over

[mg262]

@MOMonster has very kindly sent me both suggestions and a large amount of sample data ... once I digest and analyse those I should hopefully be able to write something that can deal with different kinds of samples (in a way that the above didn't due to lack of test clips). So I would advise people to wait rather than using that version... it should be treated as a rough proof that this method might be going somewhere rather than a finished reverser.

@insamedesio: thank you again for the offer; I may take you up at a later point, but not immediately as dealing with peculiar blends is an almost completely unrelated problem to this one...
____________________________________________________

All right... the explanations I promised above. Its going to come in two parts, the first of which is quite general and consists of the results of the pattern analysis I have performed to date and the second of which details how my filter functions.

Note: This is taking me quite a while to write (because of all the pictures it requires), so it will get edited in here section by section... I hope that no one finds that too distracting. I will indicate when it's finished! If I'm not being clear enough at a given stage, please poke me ASAP to minimise the amount of rewriting necessary.


Pattern Analysis

Unconverted Pattern
All the images and analysis here will describe the input in terms of frames but the output in terms of bobbed fields. For example, a (synthesised) clip which I would normally show like this:
http://people.pwf.cam.ac.uk/mg262/posts/blend/frame.jpg

Looks in terms of (bobbed) fields like this:
http://people.pwf.cam.ac.uk/mg262/posts/blend/basic.jpg
I could represent that like this:
A A B B C C D D ... (where A,B,C,D represent different pictures)

Although this may seem confusing, it does seem to be the natural representation, as I hope will become apparent when you read the rest of this. The easiest way to think of it is perhaps to all the input not in terms of frames but in terms of pictures, each of which persists for a length of time. So looking at the last diagram again, we can see that there are 5 pictures, each of which persists for 2 fields:
http://people.pwf.cam.ac.uk/mg262/posts/blend/basicN.jpg


Telecine
Another simple example is given by telecine, or more precisely by 2:3 pulldown:
http://people.pwf.cam.ac.uk/mg262/posts/blend/telecine.jpg
I could represent that like this:
A A A B B C C C D D ...

In other words we have a picture that persists for 3 fields, then one that persists for 2, then one that persists for 3, etc:
http://people.pwf.cam.ac.uk/mg262/posts/blend/telecineN.jpg

Hopefully, so far so good. Now comes the blending...


A Simple Blend
Consider this:
http://people.pwf.cam.ac.uk/mg262/posts/blend/4to5.jpg
I could represent that like this
A A AB B B C C CD D D ... (where AB represents a blend between A & B)

In this case, all the blends are 50:50. I hope that the following doesn't seem unnatural:
http://people.pwf.cam.ac.uk/mg262/posts/blend/4to5N.jpg
The idea is that the first picture was around for half-a-field's worth of time, and the second picture around for half-a-field's worth of time, so that the output field ended up being a 50:50 average of the two pictures.

Temporary note for the impatient among you... you have probably guessed that thick black 'picture divisions' in the above pictures correspond to the white lines in the (computer-generated) patterns I posted some way above. Here are patterns from @alain2's first sample (GitS) again, first at 5 fields/line

http://people.pwf.cam.ac.uk/mg262/posts/blend/gts large.jpg

... then at 25 fields per line...

http://people.pwf.cam.ac.uk/mg262/posts/blend/gts.jpg

It should hopefully be apparent from the latter picture that there is a very clear pattern to the blend weights, which I will describe + explain the origin of soon...

[MOmonster]

@mg262
thanks for your explanation. Good that I could help you a little bit. I just think that your function has great potencial. Thanks for your advise, nevertheless if I have a little bit more time I will test your function. Four eyes see more than two. ;)
@backwoods
good to hear, that my function works not so bad for you.

[insanedesio]

@mg262:
That's what I thought, but wasn't sure. When I first went through it I didn't really pay attention (didn't know much) but to the best of my recollection it was undeed very unpatterned... and very, very annoying and hard to get rid of. The Kenshin I've got is NTSC R1, and I've been hoping to be able to get the R2 somehow to take a look at that.

Oh, well. Let me know if/when you want it.

[mg262]

Well, I want to get this finished but I don't have the energy to keep generating pictures, so you'll have to concentrate a little harder! This post is not too important; it's a technical detail that should be stated but can be skipped. The next post is important.

Going back to the pictures above, suppose we have something like this:

A B BC C D

where the middle frame is a blend of the two surrounding clear frames. Suppose the middle frame is 20% of B and 80% of C. Now it is pretty clear that the divider for this frame should go either 20% or 80% of the way along. Which is it? Well, since the picture is much closer to this:

A B|C C D

Than to this:

A B B|C D

It should be clear that the divider is closer to the left than the right -- so the divider goes 20% of the way along from the left. So the pictures persist for the following number of fields:

A 1
B 1.2
C 1.8
D 1

[mg262]

Now to the real stuff. Back to Alani2's GiTS sample:

http://people.pwf.cam.ac.uk/mg262/posts/blend/gts.jpg

If we try to describe this in terms of blends and clear fields, we get something that looks quite like a pattern of period 25 but with a lot of breaks -- in short, a mess. (look back at the table in this (http://forum.doom9.org/showthread.php?p=663912#post663912) post.) Trying to find the meta-pattern directly doesn't work.
If we describe it in terms of pictures, each persisting for a length of time, we get this:

Picture persists for ~1.6 fields
Picture persists for ~2.6 fields
Picture persists for ~1.6 fields
Picture persists for ~2.6 fields
Picture persists for ~1.6 fields
Picture persists for ~2.6 fields
Picture persists for ~1.6 fields
Picture persists for ~2.6 fields
Picture persists for ~1.6 fields
Picture persists for ~2.6 fields
Picture persists for ~1.6 fields
Picture persists for ~2.4 fields

With NO pattern breaks, except where they are caused by manual cutting. (Note that this is almost a 2:3 pulldown pattern. There is a deeper pattern underlying this which I will come to below.)

If you add up the above, you get 12 pictures every ~30 fields, which is almost exactly what we expect. What we expect is exactly 12 pictures every 29.97 fields, i.e. 12.012 pictures every 30 fields. And in fact, looking back at that picture you can see that every 25 fields the pattern slides forward by a tiny amount. This is because all the durations quoted above are slight underestimates. The exact quantities are:

Picture persists for 1.6016 fields
Picture persists for 2.6026 fields
etc.

So the moral of the story is that looking at picture durations removes a lot of "counterfeit" pattern breaks.

In the next post I'm going to describe another pattern that looks even more irregular and show why it isn't -- and also explain why that 2.4/2.6 alternation is more natural than it looks. But for the moment I just want to say a couple of things about the method used by the filter.
1. The blend reversal method needs blend weights.
2. It has very little in common with the existing methods which replace blended fields with other fields; this is because I've substituted maths for video intuition.
3. That means that trying to make it replace blended fields (rather than reconstructing them) is not a simple change.

[foxyshadis]

I'm not really sure from the descriptions given in this thread, but I'm looking for a way to unblend 24->30 telecine blends. My source randomly switches between
c c c b b
and
c c c c b
So I'm pretty much giving the first type up as lost to posterity, I'll take a blended frame out of that. (unblend gives that, restore24 doubles the # of blends, and restorefps turns every frame into a 2/3-way blend.) The source isn't terrifically clean even after filtering, I guess I need to dig into some heavier smoothing filters for this to work. It's also rapidly cut, sometimes on blended frames, which I think screws up the pattern detection. There are also some sequences that turn up with huge black blobs in the unblend.

Called with restorefps(24/1.001,0.0).

I'll just upload some screenshots to illustrate. From a cowboy bebop amv.
http://foxyshadis.slightlydark.com/random/flashpics.zip

Right now I'm thinking I'll just have to pick which deblender works best for each scene and stitch them together, if I even pursue this. I'd like to mostly out of a perfectionist nature. (I already finished this video's edits a month ago, I'm just revisiting it since I know it so well now.)

Edit: Just to make clear, the above descriptions are only on problem areas; on some scenes it performs miracles, that's why I'm interested in it. :D

Edit2: More testing revealed that unblend was a better method overall. (Even if the only docs are the source code!) It is really excellent at:
a b c c d
a b c d d
but it simply gives up on:
a b b c d
a b c d d
which about 1/4 of the clip was composed of. So what I'm trying to do is come up with a good way of testing for a double-blend and reconstructing c. I think it's a little too late to be considering the maths, but I have several theories concerning overlays that might lead to something. I might have to add edgemasking and maybe motion vectors, but I want to keep it simple for now. I wish photoshop would open, I really want to test them out.

Eventually I'd add this into unblend, maybe a new mode or something. I'll probably have to start a new thread since this is really a pal thread and mine is an ntsc specific problem.

[mg262]

At the moment this is all still very experimental. Here's a rough summary:

RestoreFPS was written before I knew anything about fieldblending. It was only meant to be used on material produced by ConvertFPS, which is "frameblended" rather than fieldblended. I only really meant to make the point that consecutive blends (like c c c b b) were perfectly reversible. I kept playing with it because of the thoroughly unexpected (but very welcome) level of interest.

At the point when I only had two or three sources, I wrote ExperimentalReversalAlpha as a proof of concept, just to check the method worked. Now MoMonster has very kindly sent me a number of sources, so I can work on the problem properly; I need to analyse them and try and build a proper reversal plug-in for at least some of them. But, the analysis is rather time-consuming and, unlike most things I build plug-ins for, I have no personal use for blend reversal... so the work is going slowly -- though it will get done eventually.

I would certainly look at your clip as well, but the format is not the easiest to deal with; an AVI or VOB is much easier to deal with.

Incidentally, any feedback on my last post (comprehensible/incomprehensible/boring/interesting/relevant/irrelevant etc.) would be appreciated.

[Alain2]

@foxyshadis
Are you sure you are not talking about a simple telecine material ? If you are using a DVD source, you just need to do tfm("yourfile.d2v").tdecimate(mode=1) (you need tivtc for that : http://bengal.missouri.edu/~kes25c/)

@mg262
I have to admit it's hard for me to follow all your explanations... But it's interesting :) I hope all this hard work will lead to something good.. I really like the idea of re-creating the fields instead of selecting the good ones like in restore24, especially for crappier sources were you can have top and bottom successive fields that are all blended so no good field to select, leaving only th echoice of keeping blended fields or creating jerking clips.

[foxyshadis]

I'm familiar with telecine, this came from a telecined dvd which was blend-ivtc'd and downsampled by premier. (Thanks, adobe!) The source is still around, but the premier project, unfortunately, is not. So I'm just cleaning what I have as much as I can. The single blends are easily decimated away and the source actually does look much smoother for it.

mg, I'll send you a couple of the files I've been experimenting with deblending on later; I don't know why I ripped screenshots, but I always work on this too late at night to make any sense. =p

I've read with interest both when the thread was current and again the last few days your explanations, and they're obvious and intuitive what you're driving at. I think that your maths will also work well for this, it's just not optimized for the simpler period, patterns, and weights that ntsc blending causes, so I'm looking into that on my own.

I'm just glad I didn't get one with:
a a b c d
a b c d d
or some similar variation. Ouch.

[mg262]

Alain2, foxyshadis, thank you for the feedback.Your comments running different directions, so I settled on the following:

1. I think that it is important to finish the description of the patterns, for the following reason: if I make the filter guess/analyse the pattern (as well as the offset), it will guess a lot less well than I can and (key point) I can guess a lot less well than scharfis and others. So it will probably take patterns in a text file, like this:

1.6016
2.6026
1.6016
2.6026
1.6016
2.6026
1.6016
2.6026
1.6016
2.6026
1.6016
2.4024

(That corresponds to the example above.) It seems likely to me that there are a small number of patterns that will recur, and once these have been diagnosed and put into text files anyone can try a reversal simply by trying them one by one.

2. The exact maths used is something I can describe elsewhere. I promised MoMonster (who sent me many interesting comments and samples) an explanation for the method used to date, and then while trying to write it I realised that large parts of it will be replaced and things weren't sufficiently clear in my head... so I've been a bit rubbish about everything (also because real life has become much busier of late). When I have sufficiently clearly on paper how the new method will work, I'll outline the steps and what the maths does without actually going into the gory details, perhaps in another thread.

[mg262]

So, some promised material. As I hope was clear from the above, don't worry too much about skipping this -- it is only needed to analyse new pattern types.


The other pattern that I have analysed is very like the one above, with the following differences:

1) Blend weights are always multiples of 0.2. So the pattern basically looks like 2.6/1.6/2.6/1.6/... (instead of 2.6026).
2) Occasionally (every 175 frames or so) the pattern is broken, because a 2.6 appears where you would expect a 2.4.

(I would give you a real example, but I don't dislike you that much! All the original data I have is extremely messy, and the pattern has been extracted with chain inferences that don't illuminate much -- so I'm skipping it unless asked otherwise.)

Okay, now for an explanation of why that 2.4 (or 2.4024) crops up instead of 2.6 (or 2.6026) in the pattern described earlier and why it sometimes fails to crop up in the pattern described above.

The first thing to note is that 2.6:1.6 is almost 3:2. This makes sense because we know that NTSC->PAL fieldblended material has been put through standard telecine before being blended (a.k.a. 3:2 pulldown.)

So, we know that the fieldblending itself takes a pattern that looks like this:

2 repeats*
3 repeats
2 repeats
3 repeats
2 repeats
3 repeats
2 repeats
3 repeats
2 repeats
3 repeats
2 repeats
3 repeats
(total 30)

*(I've used the word repeats to emphasise that this pattern isn't dependent on the "picture duration" method of measuring fractional patterns that I'm using.)

And throws out something that looks like this (give or take 0.1%):

1.6
2.6
1.6
2.6
1.6
2.6
1.6
2.6
1.6
2.6
1.6
2.4
(total 25)

I've given it a bit of effort, but I can't think of a natural way to explain how the converter actually works -- its a leap of intuition/faith. The converter itself, which takes 30 fields of input and gives out 25 fields, has the following pattern (illustrated with 60 fields of input):

0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 0.8, 0.8,

0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 1.0, 0.8,
0.8, 0.8, 0.8, 0.8, 0.8,

etc. So, just as a reminder, those are the output durations of the individual input fields. When this is combined with the telecine repetition, it breaks into blocks of 1.6 (= 0.8+0.8), 2.6 (= 0.8+1.0+0.8), and 2.4 (= 0.8+0.8+0.8).

Now, of course, I've ignored that 0.1% due to the 23.976 NTSC source. It's clear what happens in the first case -- the numbers 0.8008 and 1.001 are used to make everything work. What happens in the case outlined at the beginning of this post is slightly different: occasionally, instead of

0.8, 0.8, 0.8, 0.8, 0.8,

we have

0.8, 0.8, 0.8, 1.0, 0.8,

Occurring to make up the "extra". The rules for when a 0.8 occurs in this case are very like the rules for a leap year:

Multiples of 4 are leap years, except that
Multiples of 100 are not leap years, except that
Multiples of 400 are leap years;

(shifting the pattern offset slightly)

multiples of 5 are 1.0, except that
multiples of 30 are 0.8, except that
multiples of ?180? are 1.0, except that
...

I've skimmed over the fact that top and bottom fields must not be mixed by the fieldblending; but everything can be adjusted to make sense of that...

[Mug Funky]

hehe... interesting stuff.

btw, the offer is still open if you want me to run some test signals through a standards converter. i think by now you've got enough info to know what a good test signal would be (maybe runs of 3 fields, 2 black, 1 white? probably best to use 75% white and 25% white instead, as the black and white levels may change, which could throw off your calculations).

actually, now that i think of it, test signals for standards converters are probably a good idea for calibration as well... you could put all kinds of stuff in it to see how the converter treats them (i'm curious about how well it's deinterlacer performs, as well as it's resizing).

[mg262]

@Mug Funky,

Thanks! If things have calmed down at your work then I will definitely take you up on that -- but if not, it can definitely wait. I want to avoid colourspace conversions; what colourspace does the converter use?

Also, I will fix on one long clip to test/calibrate the filter... so if there's something you particularly want reversed, for work or fun, please send a copy. Don't worry if not -- I can just extract a section from Jayce. [If you do, at least 3000 frames, size doesn't matter, any format, preferably clean/recent, at least one section with lots of change/motion, not peculiar blending and not this ecchi/hentai stuff, please!]

@all,

Here's a complete summary of everything so far:

1) I have a black box [RestoreFPS] that can reverse any known blending pattern (specified including weights) if offsets/cuts are known.
2) Considering blend weights makes some apparently irregular patterns turn out to be regular.

And one new fact: the black box also outputs a "goodness of fit" number, indicating how well the given pattern/offset seems to fit the data.

So, there is a easy way to reverse blending. Take the pattern (analysed by hand), and use the black box to try reversing it with each possible offset (or, say, 1000 different offsets), and picking the offset that gives the highest "goodness of fit".

By using some maths, this can be done much faster than it sounds (although a fair bit slower than real-time). A particular programming trick will also allow cuts to be treated sensibly.

NB this method makes no use of bobbing whatsoever, and handles any number of consecutive blends. If there are no consecutive blends, we can calculate the blend weights using bobbing*, and use the weights to guess the offset -- this can probably be made to work in real time.

*I did this above, and it's part of what I promised to explain... more soon.

[mg262]

As I said just above, in the case where there are no consecutive blends, you can use some fairly simple maths to calculate the blend weights. The method I used in ExperimentalReversalAlpha relies on properties of the SSD of two pictures (sum squared distance -- take the 720*576 luma differences for corresponding pixels, square them all, and add them up -- the smaller this number is, the more similar the pictures are). It works like this:

First bob all the fields. Now, suppose you have a couple of fields A and B, and you want to test whether a third field C is a blend of A and B. We draw a triangle like this
http://people.pwf.cam.ac.uk/mg262/posts/blend/write-up/triangle.png
where each side length is equal to the square root of the SSD of the two fields joined by the side.
http://people.pwf.cam.ac.uk/mg262/posts/blend/write-up/distances.png
So in this picture you can see that field C is much closer to field A than either is to field B -- and therefore that field C is much more similar to field A than either is to field B.

Now, SSD pictures like this have some useful properties. First, any noiseless blend of Field A and Field B will lie on the line between those two fields.
http://people.pwf.cam.ac.uk/mg262/posts/blend/write-up/blend.png
What's more, if e.g. the blend is 80% of A + 20% of B, then it will lie 20% of the way along the line from A to B.* In other words, the fraction of the distance along the line that the blend lies gives the blend weight.

*(Not 80%, because it's much more similar to A then B.)

So, we have a good way of
i) determining whether a field C is a blend of field A and field B
-- just check if it lies close to the line between field A and field B
ii) (if we decide it is a blend) finding the blend weight
-- find the closest point to C on the line between field A and field B, here marked X:
http://people.pwf.cam.ac.uk/mg262/posts/blend/write-up/project.png
and calculating the blend weight from X. (In the picture shown, about 70% A, 30% B.)

So that is the core of the method I used for blend detection and calculating blend weights.

[MOmonster]

Thanks for the good explaination. Do you use thresholded differences, I mean no regard of differences under 1 or 2 values, or do you use the simple difference? This is very, very similar to the method I use for Crestore, but the simple LumaDifference() is not accurate enough on frames with very little motion, because the differences can here just be bigger because of the compression, so my function fails sometimes on too low motions.

[mg262]

I square the per-pixel differences and then add them together (SSD). Squaring things has the effect that small differences have a (proportionally) much smaller impact than large values. For example, suppose we have the following luma differences between two 3 x 4 frames:

12 12 12
0 0 0
0 0 0
0 0 0

The SAD (sum absolute luma difference) is 36; the SSD is 3*(12*12), or 432.

Compare that with a situation where we have these luma differences:

3 3 3
3 3 3
3 3 3
3 3 3

The SAD (sum absolute luma difference) is still 36; the SSD is 12*(3*3) = 108.

The average luma difference in both cases is just the SAD divided by the number of pixels, which is 3. So looking at average luma difference (which is, I think, what LumaDifference produces?) treats these two cases as identical. But the SSD says that in the first case the difference (108) is much less significant than in the second case (432)... which makes sense because small values like 3 could well be caused by noise but larger values like 12 are much less likely to be.

Hope that answers the question? Do feel free to ask more!

[MOmonster]

Ok, yes this I can´t do with Lumadifference.
Another question: When it get the blendweight, does your filter correct it to a fixed value? I mean real 24p to NTSC to Pal conversions (normally it is 23.976 fps, so much stranger weights, but I think also fixed) has for example only these blendweights: 25%, 40%, 60% and 75% (I think it was so for real 24p, but I can be wrong).
So or so, the blendweights of the current pattern should be really much the same then for the last pattern (for the same lenght with no edits and so on), so it can be useful if the pattern for example says 25% and the math says 23.5%, that we use the idealistic weight.

[mg262]

The pattern I tried it on didn't have a small set of possible weights, so it didn't do that -- but the final product certainly will. The results of ExperimentalReversalAlpha led me to conclude that trying generalise reversal in the absence of any knowledge about the pattern was not going to work for all patterns. Hence I decided that the filter would be fed with the pattern, and it would just try it with every possible offset (i.e., if you like, try it at every possible starting frame), and pick the best fitting offset. So not only will it only use blend weights that occur in the pattern, but also the order in which those weights ago will always be a legal one. In other words, weights are only ever used to guess the pattern offset, and are never fed directly into reversal.

Tangential: I wish I was dealing with real 24p! If I was, I would probably have everything finished by now. 23.976 is a nightmare because the decimation ratio ends up being something like 2997:1250... and decimating like that while trying only to look at the local neighbourhood of the current frame is nasty. I will elaborate on that in a bit.

Edit:Ok, yes this I can´t do with Lumadifference.I will look at exposing the blend detection method I use so you can use it in scripts.

Edit 2: I was deliberately vague when I wrote thisi) determining whether a field C is a blend of field A and field B
-- just check if it lies close to the line between field A and field B... I tried five different measures of "nearness" to detect blends, all based on some properties of the triangles above, and I actually found that the best measure was the angle at field C. If field C lies near the line, then the angle is large -- near 180, whereas if it is far away from the line, the angle is much smaller.

To access the blend angle you will need both the latest version of this plugin:
ReverseBlend, 22 October 2005 (http://people.pwf.cam.ac.uk/mg262/posts/blend/ReverseBlend_22Oct05.dll)

and another plugin from which I have borrowed some functions:
Cel Background, 22 October 2005 (http://people.pwf.cam.ac.uk/mg262/posts/Background/CelBackground_22Oct05.dll)

Then try something like this:
Loadplugin("ReverseBlend_22Oct05.dll")
Loadplugin("CelBackground_22Oct05.dll")

#source
bob() #use the bobber of your choice
if (blendangle().isabove(100) ,\
then = subtitle("Blend") ,\
else = subtitle("Not Blend"))The subtitle(...) parts can be replaced with anything you like (n.b. they are implicitly using last); 100 can be increased to detect fewer blends or decreased to detect more.

(Please don't mention a 17 letter word at this point... ;) . This method has its advantages.)

Edit 3: Please let me stress that this is only for the case where you don't have consecutive blends. As noted above, there is a different, slower, method which will work in all cases. I should also note that this method sometimes picks up very slight blends (e.g. 95%/ 5%) which can be hard to note by eye. If it is useful, I can expose the blend weight as well, so you can condition on e.g. the weight being at least 10% of each field.

I created a version with blend weight as well, but I have to go out right now so it's really not tested... use at your own risk (http://people.pwf.cam.ac.uk/mg262/posts/blend/ReverseBlend_23Oct05.dll) ... or wait until tomorrow and I will test it! Sample script:

#load your clip
bob
source = last
Anglecriterion = blendangle(source).isabove(100)
weight = blendweight(source)
weightcriterion = And(weight.isabove(0.1),weight.isbelow(0.9))
criterion = And (Anglecriterion, weightcriterion)
if (criterion,\
then = source.subtitle("Blend"), \
else = source.subtitle("Not Blend"))

[foxyshadis]

:thumbzup:

I just wanted to point out the the reverseblend is actually at 23Oct05 (http://people.pwf.cam.ac.uk/mg262/posts/blend/ReverseBlend_23Oct05.dll) for those who get 'no blendweight function'.

It seems to consider a few sequences blends that aren't, mostly fade ins/outs and slow scrolls, but I imagine this is just a matter of tweaking the settings. I'm going to play around and see what happens.

Well yes of course it would get fades, those are just prolonged blends. Duh. And I guess the weight.isabove/below parameters can help keep it to known blendweight areas. (50/50 in my case but the crappiness of the source necessitates slightly wider range.) A few trouble scenes still get by with .3-.7, one of which is frames 97-98 of sample clip 2 in my thread, both should be 50/50 but I guess the motion in the second is too small. I'll expand the weights a bit and see what happens.

[Mug Funky]

to throw a spanner into the works, some standards converters have scenechange detection where they attempt not to blend across scene cuts (this probably doesn't always work in practice), and probably means the pattern "phase" resets itself a lot.

@ mg262: about the standards converter and test signals, etc. it works on SDI inputs (it's got the option to input from composite, YC and component, but we never got that add-on), so we're basically talking yuy2 here (except in 10 bits). however, what the decklink card or final cut pro (or premiere if i put the card in a PC) actually does to the output remains to be seen. there could well be some tweakage of colours and black/white points. this is why i feel a test signal should stick to between 25% and 75% white rather than 0% and 100% - we need the headroom to allow for things out of our control.

i think the most versatile test patterns should be generated through an avs script - we can do colourbars either with the internal generator or with stackhorizontal/vertical and blankclips. stuff can be moved around with animate and crop/resize. plus the test clip would be a text file rather than a big hulking avi file.

another thing - i couldn't find a pertinent bit to quote, but you mention you're assuming a "noiseless" blend between fields. the converter is digital, so no matter how noisy the source is, the actual blending will not have any noise (except rounding error, but that should be very minor). however, after mpeg-2 compression a fair bit of noise could be introduced. this _shouldn't_ effect blend detection (though if the picture is encoded as progressive there'll possibly be ringing between fields that may actually throw the average luma out by a fair bit. no sane person would encode standards-converted stuff as progressive, but not everyone is sane), but it may have an effect on removal.

[mg262]

I've finally thought of away to articulate the difficulties due to decimation with weird ratios (which are the main reason I haven't implemented anything yet). Suppose we are decimating with a ratio 2997:1000 -- exact numbers don't matter, as long as they are large. So we are outputting 1000 unblended fields for each 2997 fields of input. Now:

The first 500 unblended fields may come from either 1498 fields of input or 1499 fields of input. (NB 2997/2 = 1498.5.) Both are possible -- it depends on the pattern offset. You can determine which happens by just looking at the first 1500 or so fields of input. Now suppose the user requests output field 550 (or so). The filter has to know whether the first 500 fields come from 1498 fields of input or 1499 -- otherwise it doesn't know which field number to start reversing from. E.g. we might find:

If the first 500 unblended fields come from 1498 fields of input,
-- field 550 should be computed from input fields 1648 onwards

If the first 500 unblended fields come from 1499 fields of input,
-- field 550 should be computed from input fields 1649 onwards

(This is simplified, but I think this essential problem really occurs.)

Now if the filter doesn't prescan the entire clip, and seeking is allowed and reasonably fast, we have a problem -- the filter should only examine a limited number of fields (e.g. 1600 to 1700), but it needs to calculate something that is really a property of fields outside this range (0 to 1499).*

*The filter can try and guess which of 1498/1499 happens by using the information between 1600 to 1700 -- and actually in many cases it can make a pretty good guess -- but if we do this the behaviour of the filter starts to depend on the order in which fields are accessed, which is something I would prefer to avoid in this case.

There are a few solutions to this. First, we could allow prescanning of the entire clip -- but feedback above suggests this is not something people are comfortable with. Second, we could disallow seeking -- i.e. impose the requirement that fields must be requested in order. In this situation, the filter would probably have some start-up overhead (i.e. a long pause or dummy frames when you start loading/playing), and then display subsequent fields fast... perhaps even in real-time. Third is to try to modify things so the first 500 unblended fields are forced to come from 1499 fields of input -- by duplicating/dropping a field in a very few places (1 drop, 1 duplication per 2997 fields -- maybe 2 of each to preserve field order). Again this should be fast, and it also allows seeking.

Trying to pull the right solution out of this is what is slowing me down. It's not just a matter of preference -- I think that if I think about it hard enough, one of these (probably the second) will turn out to be right for some subtle reason...

Mug Funky,
Brilliant -- thank you very much. I was definitely thinking in terms of a script -- I will write one when I'm a bit more with it than at present. But I will also keep it to something that will lossless-compress really well (e.g. flat colour in each 8 x 8 block) so the output can be dealt with sensibly. I'll keep well within the extremes of the luma range.

Pattern phase (= offset) is something I definitely plan to deal with in the filter, since it can also occur due to simple cutting or telecine change (these two cases will hopefully be separated and dealt with appropriately). But cuts actually make the above issue even more complicated...

[Didée]

Just taking the rope and cutting it into equal parts is dicey. Cling yourself to the rope and pull forward steadily.

[jmac698]

this is a fascinating discussion, you've really made some progress with hard work. Brilliant! I may take this on and help out...

[jmac698]

I tried restorefps, it simply crashes. Also trying the blenddetector, oct23 version, with the Angle() function. There was never a value displayed.

[jeffy]

Does anyone have the CelBackground dll download link, please? Thank you.

http://people.pwf.cam.ac.uk/mg262/posts/Background/CelBackground_22Oct05.dll

referenced here:
http://forum.doom9.org/showthread.php?p=726948#post726948

[WarpEnterprises]

I have and will post it at the filter page. I didn't add it because of the complete lack of documentation.
(but not the usage script, maybe someone can post it)

[jeffy]

Thank you, WarpEnterprises!

[Prettz]

Does anyone have the CelBackground dll download link, please? Thank you.

http://people.pwf.cam.ac.uk/mg262/posts/Background/CelBackground_22Oct05.dll

referenced here:
http://forum.doom9.org/showthread.php?p=726948#post726948
I have and will post it at the filter page. I didn't add it because of the complete lack of documentation.
(but not the usage script, maybe someone can post it)
Cool. I was about to post that the last version I have is named "CelBackground_23Sep05B.dll". I think I must have abruptly stopped visiting this thread after that since I have nothing dated from October.

[WarpEnterprises]

I'm sorry, I didn't take care of the date, so I think I too only have the September version...

[foxyshadis]

you guys are lucky:

http://foxyshadis.slightlydark.com/random/CelBackground_22Oct05.dll.zip

[jeffy]

you guys are lucky:

http://foxyshadis.slightlydark.com/random/CelBackground_22Oct05.dll.zip
:( 404 error
I am also searching for ReverseBlend, the two versions 22nd & 23rd October 2005, do you please have them?

Thank you for your help.

[foxyshadis]

I didn't notice that the firewall at the office blocked the upload! So sorry, fixed now. Also:

http://foxyshadis.slightlydark.com/random/ReverseBlend_23Oct05.dll.zip

[AnnaFan777]

Those of you reading this thread for the first time might want to start at this post (http://forum.doom9.org/showthread.php?p=691286#post691286), which is where the attack on fieldblending starts for real.
____________________________________________

RestoreFPS (http://people.pwf.cam.ac.uk/mg262/posts/RestoreFPS.dll)

Brief description:
Reverses the kind of blending generated by ConvertFPS, restoring original framerate.

Search keywords: ConvertFPS, FPS, framerate, restore, reverse, like unblend, deblend, restore24.

Full description:
RestoreFPS(clip, float fps, float phase) - takes a YV12 clip which has an 'underlying' frame rate of fps but a higher actual frame rate due to blending, and reverses the blending to restore the original frame rate. phase is a number between 0 and 1 which specifies the relative displacement of old and new clips (see example below).

The framerate to restore should be less than the current framerate, and more than half of it. (So restoring from 24 back to 25 and from 24 back to 11 are both illegal.)

The method used is described here (http://forum.doom9.org/showthread.php?s=&threadid=89266).

Examples:
In order to test the filter we need to generate a clip using ConvertFPS. A script like this:

bicubicresize(4*60,3*60)
selectevery(200,0)
converttoyuy2
assumeFPS (24/1.001)
convertFPS (25)
converttoyv12
overlay(crop(0,0,-0, 26). showframenumber)

will do the trick. Some frames from this:

http://people.pwf.cam.ac.uk/mg262/posts/convertfps.jpg

Now running

RestoreFPS(24/1.001, 0.00)

produces a stream like this (do look at the messed up 'frame numbers'):

http://people.pwf.cam.ac.uk/mg262/posts/convertfps restored.jpg


In this case, the start (in time) of the input clip corresponds exactly to the start (in time) of a frame of the original clip. In other words the situation looks like this:

http://people.pwf.cam.ac.uk/mg262/posts/fpsconvert.jpg

That won't always be true -- if we take the 'convertFPS' clip above and trim it (say add Trim(10,0)), then we have something more like this:

http://people.pwf.cam.ac.uk/mg262/fpsconvert2.jpg

Dealing with this requires us to guess the amount by which the converted clip is 'slid' with respect to the original, i.e. to guess the phase parameter. I used this script to help with this:

Function phase(clip c, float phi)
{
c
RestoreFPS(24/1.001, phi) #set appropriate frame rate

#trim here if desired

a=selectevery(6,0)
b=selectevery(6,1)
c=selectevery(6,2)
d=selectevery(6,3)
e=selectevery(6,4)
f=selectevery(6,5)

stackhorizontal(a,b,c,d,e,f)
Reduceby2()
stackvertical(last, trim(1,0), trim(2,0), trim(3,0), trim(4,0), trim(5,0))
Trim(0,-1).Loop(0,1000)
subtitle(String(phi, "%f"))
}

#source clip here
animate (0, 100,"phase", 0.0, 1.0)
trim(0, 100)

Save the output of this, open it up in VirtualDub, and drag the slider around to find a phase that looks correct. (You may get an unusual effect in the first few frames of the clip, because of an edge effect.)

Here's a real example to finish up with:

Source (from VHS):
http://people.pwf.cam.ac.uk/mg262/posts/blends.jpg

Restored:
http://people.pwf.cam.ac.uk/mg262/posts/blends removed.jpg

(Actually, this example doesn't seem to work exactly like ConvertFPS on a wider scale, but leave that aside for now.)

Notes:
- If you're dealing with a clip in which fields (not full frames) are blended, separate them and process them separately; I think the phase parameters for the top and bottom fields will differ by 0.5 (though I'm not at all sure).

- This isn't optimised; there are several ways in which I could speed it up, and I will if anyone finds it useful.

- If you get the frame rate slightly wrong, this filter will work in a small region and get worse and worse as you drift away from it.

- It will not deal with cuts. You will need to take each cut section and find a phase for it separately. (If the sections are too short, this method may not be of any use.) On the other hand, there are some ways to estimate the phase automatically (esp. autocorrelation), and cuts could be detected by the change in phase.

- Although I've only set it up to reverse ConvertFPS type blends, it will work perfectly well on any regular blend pattern, including the types (http://home.arcor.de/scharfis_brain/ExotischesInterlacing/#2.3) mentioned by scharfis_brain in this thread (http://forum.doom9.org/showthread.php?s=&threadid=89266). The hard part is analysing the blend type -- I have some scattered thoughts but this post is too long already!

By the way, I'm really sorry I trailed off/disappeared at the end of that last thread... real life got in the way. I'll try and do better with this filter (assuming anyone finds a clip it works on!)

Thank you, but the demo pics are all gone.

and there's no doc in the DLL package,
So , can someone revive this threat ?

[zee944]

I have a badly encoded video, this one:
http://www.badongo.com/file/12382006 (sample, 12 Mbytes)

I'm just assuming a 25 fps PAL master was used and blended to 29.97i. (Or perhaps fieldblended to 23.976 then telecined.) A lot of frames can only be found as blended. I know from other threads RestoreFps() would probably be the best solution to this, but I can't get it to work.

First, this phase-finder script doesn't work:

Function phase(clip c, float phi)
{
c
RestoreFPS(24/1.001, phi) #set appropriate frame rate

#trim here if desired

a=selectevery(6,0)
b=selectevery(6,1)
c=selectevery(6,2)
d=selectevery(6,3)
e=selectevery(6,4)
f=selectevery(6,5)

stackhorizontal(a,b,c,d,e,f)
Reduceby2()
stackvertical(last, trim(1,0), trim(2,0), trim(3,0), trim(4,0), trim(5,0))
Trim(0,-1).Loop(0,1000)
subtitle(String(phi, "%f"))
}

#source clip here
animate (0, 100,"phase", 0.0, 1.0)
trim(0, 100)

It only works if I delete the Trim(0,-1).Loop(0,1000) line, but then it becomes suspicious something isn't right. Even then I don't know what to look for - a moment where all the 6 frames look perfect? I've played a LOT around the phase value in a different script, but couldn't find a good value.

LoadPlugin("Restorefps.dll")
MPEG2Source("Twin_Dragons_Joy_Sales_VideoFile.d2v")
Bob()
RestoreFPS(47.952,[<"phase", 1, 1000, 331>]*0.0033) #set appropriate frame rate

The example pictures are long gone too.

Could someone help pointing me to the right direction? This is the only source worldwide of this version of the movie, and I've no idea what to do now. Thank you in advance.

[DoctorM]

Sorry to bump an old one, but does anyone have a working link for RestoreFPS or a suggestion for a newer filter that does the same thing?
Edit: Sorry, found it on AviSynth's Wiki: http://avisynth.org/mediawiki/RestoreFPS

[Emulgator]

SRestore.

[Didée]

SRestore.
No. Not even remotely.

some_video_clip_in_24fps

ConvertFPS(25.000)

Now try SRestore. Big big fail, simply because it is not made for this kind of blending. RestoreFPS is.

[DoctorM]

Just starting out, but RestoreFPS seems to be working great. My source appears to have been 24p -> 25i (with blended fields) -> 29.97i (with more blending).
I'm getting some pretty good results stacking restoration filters: RestoreFPS(50).SRestore(23.976). I'll toy with it a bit more later.

[Emulgator]

Hm, true, I should have used it on such source before suggesting...

[DoctorM]

Okay, it was crazy, but I figured my bizarre video out.

The source underwent: 24fps film to 25fps PAL video (by speed up) to 29.97i video (by blending fields).

To repair:
yadif(mode=1) #produced cleaner frames in this case
restorefps(50,1)
selectodd()
assumefps(23.976)

Never seen it done before. Very bizarre.

To be fair about two thirds of the way through the even frames became the good ones. I fixed that by splitting the video and adjusting the selectodd.

Thanks for the very useful RestoreFPS!

[ceth]

[...]I can't get it to work.
First, this phase-finder script doesn't work:
[...]
It only works if I delete the Trim(0,-1).Loop(0,1000) line, but then it becomes suspicious something isn't right. Even then I don't know what to look for - a moment where all the 6 frames look perfect? I've played a LOT around the phase value in a different script, but couldn't find a good value.
[...]
The example pictures are long gone too.
Could someone help pointing me to the right direction?
Hello,

Same story for me.
And I'm new to this. I managed to fix a doubleblend with srestore but I don't understand the process of using RestoreFPS.

Could someone help with some more noob-friendly explanations about using RestoreFPS ?

To begin, as for zee944 I can't make the phase-finder script work. Using the full script result in only 1 frame (made of 6x6 frames) and I only get more by commenting the Trim(0,-1).Loop(0,1000) line :confused: