Bob filter with no resize

[525/60]

Unfortunately, BobInPlace has to be called like this: BobInPlace.SeparateFields.AssumeFrameBased.AssumeTFF
That is bad. Separating the fields within the filter would be better.

The other thing that is bad is that it has no assembler code in it; it is slow.

Other aspects of its primitive nature are based on a deliberate choice to minimize filtering in order to maximize sharpness.

Basically, the idea of this filter is that a simple average between two lines is effective in siting the image properly and because the frames derived from the unfiltered top fields are perfectly sharp, temporal filtering is more effective than additional smoothing of the frames derived from the bottom fields.

Originally, BobInPlace was written as an alternative to Bob --> Resize --> Reinterlace of interlaced material that must be resized, but I haven't successfully written a DeBobInPlace filter yet.

The question being: 'Why let bob resize when the user is just going to resize anyway?'. Well, that question is valid for nearly any bob, isn't it?

Instead of moving both fields down and up 1/4 line, BobInPlace leaves the top field untouched and takes the average of every bottom field's line with the bottom field's line that procedes it. This simple approach maximizes both sharpness and flicker. But wait! Divx' attempt to minimize differences between consecutive frames for compression acts like a temporal smoother, removing the flicker entirely. So when you step through the avs file with VirtualDub, you see a sharp frame alternating with a blurry frame and when you play it in VirtualDub you see a little flicker, but the eye cannot see the blur; and after an encode, the flicker is gone too.

So: Why should a bob filter do a resize anyway? Won't the user almost always do a resize subsequently? And won't the two resizes reduce sharpness unnecessarily?

Stephen
7091

[scharfis_brain]

for TFF video:
assumetff().bob(0,1).assumetff().separatefields().selectevery(4,0,2)

for BFF video:
assumetff().bob(0,1).assumetff().separatefields().selectevery(4,0,2)

I don't see, why one should EVER do this, cause it throws away half the vertical resolution of the video and flickers like hell.

Also, a deinterlacer does NOT resize. It just recreates missing lines in a spatio-temporal matter.
bob() is an implementation to deinterlace using a resizer.

[wonkey_monkey]

for TFF video:
assumetff().bob(0,1).assumetff().separatefields().selectevery(4,0,2)


Doesn't that just equate to a single separatefields()? And did you mean to keep assumetff in your bff example?

David

[525/60]

@scharfis_brain
Bob doubles the height of each field. Your example is what I am refering to, but what happens when/if you want to change the aspect ratio. Haven't you used bob to resize once to double the height and then resized again? The only way that that is acceptable is if you resize only the width and not the height when adjusting for the aspect ratio.

@all
BobInPlace passes through the top field (doesn't change it at all) and aligns the bottom field with the top field, rather than moving both fields toward the middle as it were. So the fields are still aligned, but every other field is not filtered at all.

The greatest advantage of this method, perhaps, is that you can use temperal filtering to sharpen the static areas of the bottom field. The flickering disappears in the avi file automatically if you compress the video in any way, despite the fact that you can see flickering in the avs file. It is interesting that the flickering disappears, no?

Maybe I should show you some of the code, to help clarify what it does:

for (int y = 0; y < (heightY)/2; y++) {

for (int x = 0; x < row_sizeY; x++) dstpY[x] = srcpY[x];
srcpY += src_pitchY;
dstpY += dst_pitchY;

for (int x = 0; x < row_sizeY; x++) dstpY[x] = (srcpY[x]+srcpY[x-2*src_pitchY])>>1;
srcpY += src_pitchY;
dstpY += dst_pitchY;

}


I changed one thing about it, making it pseudo-code.
Oh, how I would love to know how to write that in MASM.

Stephen

[wonkey_monkey]

525/60, I thought I understood when you said:

Instead of moving both fields down and up 1/4 line, BobInPlace leaves the top field untouched and takes the average of every bottom field's line with the bottom field's line that procedes it.

But your latest post has lost me entirely :) What do you mean by:

aligns the bottom field with the top field

?

David

[scharfis_brain]

@525/60:

my script-line EXACTLY does what you claim your filter does.
passes through the first field unaltered and delivering the second field 'aligned' (I call it blurred) vertically to the first field by cubic interpolation.

Of cause, I abuse a resizer here, but that does not make a difference in the result.

Also I doubt that interlace flicker will go away just by compressing the footage.

if you take my script from above and just replace bob() of this line with yadif(mode=1) or even mcbob() you'll notice that most of the flicker will be gone AND the 'aligned' fields stay relatively crisp.

There is no such thing like a simple, fast and artifactfree deinterlacer.
There must be a reason, why poeple all around the world are searching for better methods to deinterlace.

[wonkey_monkey]

My understanding was that it blurs the bottom field not with the upper field, but with the previous bottom field. I wrote a script which implements this, but it does still look terrible - full of combing, as you'd expect. Static areas look fine, but then, they look fine in the original interlaced video too.

The fact that this combing disappears when you encode (although it didn't for me) is not a good thing - it means you're passing duff video to the encoder and it's having to throw it out. I think it's a pretty sure thing that you can never improve video by compressing it.

David

[525/60]

Also I doubt that interlace flicker will go away just by compressing the footage.No, no, I absolutely mean that I tried it out, and to my surprise the flickering was gone in the avi file, though it was clearly there in the avs file. Again, the mere act of encoding acted as a temporal filter.

my script-line EXACTLY does what you claim your filter does.
passes through the first field unaltered and delivering the second field 'aligned' (I call it blurred) vertically to the first field by cubic interpolation.I take your word for it, except for one thing. I am refering to not filtering the top field at all and therefore the height of the resulting frame will be half that of the input.

It is equivalent to bob(0,1,240) or bob(0,1,288) for PAL.

For bob it probably does not matter if the height if each field is doubled, but for BobInPlace the advantages are:

1) Every ofther frame in the output is based on the top fields of the input unchanged.

2) The bottom fields are processed by taking the average of consecutive lines of the bottom fields.

3) The alteration of sharp frames derived from the top fields followed by blurry frames derived from the bottom fields and the consequent flickering is easily removed by causing one frame to conform more to the other frame. That is a simple temporal denoising that many encoders use to improve compression.

Maybe bob filters do more than they need to do. Maybe flickering tends to get compressed out anyway. Trust me, it works or at least I look forward to someones input on how it works after he has tried it. It is not easy to post an example here. All I know is it worked for me, but it took way too long.

It would be nice if someone could suggest a way to filter every other frame to conform to the previous frame, since in this case it is known which frame will be sharper.

Stephen

[525/60]

@davidhormanI wrote a script which implements thisShow me the script

Stephen

[wonkey_monkey]

Okay, bear in mind I wrote this based on that one line of yours:

Instead of moving both fields down and up 1/4 line, BobInPlace leaves the top field untouched and takes the average of every bottom field's line with the bottom field's line that procedes it.

source=mpeg2source("kirsty.d2v").assumetff

sep=source.separatefields # t1,b1,t2,b2...

blur=overlay(sep,sep.trim(0,1)+sep,opacity=0.5) # t1+t1,b1+b1,t1+t2,b1+b2
blur=blur.trim(1,0) # b1+b1,t1+t2,b1+b2,t2+t3

interleaved=interleave(sep,blur) # t1,b1+b1,b1,t1+t2,t2,b1+b2,b2,t2+t3

switched=interleaved.selectevery(4,0,1,3,2) # t1,b1+b1,t1+t2,b1,t2,b1+b2,t2+t3,b2

switched.weave


To explain: blur blurs together adjacent top and adjacent bottom fields, based on my understanding of the above quote. The rest of the script interleaves these blurred fields with the original fields (both top and bottom) in the correct order to be weaved, so the result is full frames made of, alternately:

a) an original top field, and two averaged bottom fields, one from the same frame as the top field, and one from the previous frame
b) an original bottom field, and two averaged top fields, one from the same frame as the bottom field, and one from the next frame

In the notes, + means averaged (if the notes make any sense at all).

This script outputs full progressive frames, doubling the framerate.

[quote]That is a simple temporal denoising that many encoders use to improve compression.[quote]

Encoders don't, to my understanding, use temporal denoising - it's a side effect of motion compensation compression, and an undesirable (but unavoidable) one at that. A codec's purpose is to do it's very best to perfectly reproduce the input video with a limited number of bits. If you use a high enough bitrate, all your artefacts should be present. Remember the old adage - garbage in, garbage out. Whatever result you get, I think you'll get a much better one using dgbob or leakkernelbob - or, I'm pretty sure, even plain old bob.

Disclaimer: all this is a bit speculative, as I was too impatient to wait for a mod to approve your attachment. I also used Xvid instead of Divx, and if Xvid more accurately reproduces the input, artefacts and all, that's all the more reason to ditch Divx ;)

David

PS What do you mean by "conform" in the above? If you mean "motion compensate", again, this is not something you can leave to the encoder. There are several motion compensating deinterlacers.

[scharfis_brain]

1) Every ofther frame in the output is based on the top fields of the input unchanged.

2) The bottom fields are processed by taking the average of consecutive lines of the bottom fields.

This is exactly what this line does:
assume?ff().bob(0,1).assume?ff().separatefields().selectevery(4,0,2)

even field is untouched, odd field is shifted by 1/2 pixel to match the vertical position of the even field.

bob(1,0,height/2) always interpolates (blurs) EVERY field cause it shifts every field by the amount of 1/4 pixel in the correct vertical position.

But in the end it is better to at least use a motion adaptive deinterlacer such as yadif(mode=1) to feed the temporal denoiser or compressor as you get twice the vertical resolution to work with as well as non-flickering static areas.

In general algorthims without sensing for the actual video contents cannot produce a reasonable result, IMO.

[525/60]

@davidhorman
Encoders don't, to my understanding, use temporal denoising - it's a side effect of motion compensation compression, and an undesirable (but unavoidable) one at that.I'm confused. How is motion compensation compression (desirable or not) not a form of temporal denoising?

@scharfis_brain
even field is untouched, odd field is shifted by 1/2 pixel to match the vertical position of the even field.A field cannot be untouched if it is not still exactly half the height of the frame. That is why I think that a bob filter should either work the way BobInPlace does or allow the user to specify a target height to resize to as bob does do. I suppose they all have that option, I just do not know enough about them.

There must be a reason, why people all around the world are searching for better methods to deinterlace.Yes, I do agree that it is unlikely that I am on to something earth-shattering. But in practice people don't seem to use the technology to full advantage. My idea grew out of some scripts I saw people use. More than likely it is the scripts that need replacing, not the existing bob filters. Either way though, something's got to give. Writing a filter is always an exercise in finding the best way to use AviSynth.

As an example of a script that needs tweaking:

bob(0,1).assume?ff().separatefields().selectevery(4,0,2).Weave

This does not do anything unless you put a resize between bob and separatefields, yet it is better to resize directly to the desired height inside bob.

Stephen

[scharfis_brain]

have you tried out my line of avs-code?

assume?ff().bob(0,1).assume?ff().separatefields().selectevery(4,0,2)

It must be this line with assume?ff() twice and WITHOUT weave.


the bob() filter puts out height doubled fields.
but every other line of these fields are taken directly from the input and the remainline lines are averaged lines.
separatefields().selectevery(4,0,2) just separates the stream after bob() into even-field-untouched, even-field-interpolated, odd-field-interpolated, odd-field-untouched.
selectevery(4,0,2) selects then even-untouched an odd-interpolated.

[foxyshadis]

@davidhorman
I'm confused. How is motion compensation compression (desirable or not) not a form of temporal denoising?

I'd say it is, but it's backwards-only (barring some minor b-frame effect) and totally uncontrollable, working sufficiently only when the bitrate is so low that quality suffers everywhere. Then again, anyone who wanted high quality wouldn't work with a deinterlacer like this.

It might work for shaving some size off, since you're halving the vertical resolution and handing the encoder blurry, easily compressable frames every other frame, but why settle for sub-VHS resolution just to have 60fps?

[525/60]

@foxyshadis
but why settle for sub-VHS resolution just to have 60fps?I think it is comparable to VHS resolution. Because for true interlaced material, it mimicks what the camera originally recorded. If each field was originally shot at a different point in time, then there is nothing you can do but separate the fields. I suppose the argument is that fluidity of motion is a time of resolution in itself. But I don't want the discussion to become a debate on whether bobbing is a good thing to do in the first place, but rather: What's the best way to bob?

@scharfis_brain
selectevery(4,0,2) selects then even-untouched an odd-interpolated.I see now, sorry I misread it.
the bob() filter puts out height doubled fields.
but every other line of these fields are taken directly from the input and the remainline lines are averaged lines.Well, if that is how it works, then the line would do the same thing as BobInPlace.

@all
No one seems to appreciate the idea that filtering one field and leaving the other untouched followed by temperal denoising may work as well or better than filtering both fields equally at the start. The other idea is that perhaps filters should be judged by the end result after the encoding process rather than what the avs file looks like as you step through the frames in VirtualDub.

Disclaimer: all this is a bit speculative, as I was too impatient to wait for a mod to approve your attachment. I also used Xvid instead of Divx, and if Xvid more accurately reproduces the input, artifacts and all, that's all the more reason to ditch Divx I was surprised by the result using Divx, but maybe I should try using Xvid also to see if I get the same result.

Stephen

[wonkey_monkey]

Do you have the facility to upload the filter and/or some short before/after samples elsewhere?

David

[wonkey_monkey]

Doh... the mods finally approve the attachment and it's source only. Any chance of a dll?

David

[525/60]

It only requires avisynth.h and cl.exe to compile.

cl /LD BobInPlace.cpp

Try:
.NET Framework 2.0 Software Development Kit (http://www.microsoft.com/downloads/details.aspx?familyid=fe6f2099-b7b4-4f47-a244-c96d69c35dec&displaylang=en)

or Download the Visual Studio 2005 Express Editions Now! (http://msdn.microsoft.com/vstudio/express/downloads/default.aspx)

Stephen

[wonkey_monkey]

You say only as if they aren't 350mb and 70mb downloads ;)

ETA: Stumbling through the source code, I see I was wrong about what you meant in your description. As well as Scharfi's code above, it seems to be (almost*) equivalent to:

bob.pointresize(width,height/2)

One or other of the fields (it's not clear which) will pass through unprocessed, and the other will be interpolated and aligned with the first - *in your code, by bilinear interpolation, but in the above, by bicubic interpolation, which should make it a little sharper.

I think it's worth pointing out too that the apparent sharpness of the untouched field is down to aliasing - it's a picture with half the lines missing.

http://img15.imgspot.com/u/07/108/10/aaa.png

Which smaller a looks sharper, and which looks best? (oops, rule 12!)

David

[scharfis_brain]

bob.pointresize(width,height/2)

OMG. THAT is stunning easy! Why haven't I thought of it ;)

returning a reversed clear-to-interpolated order can be achieved through vertical image shifting:

bob.crop(0,1,0,0).addborders(0,0,0,1).pointresize(width,height/2)
(Probably you'll get a black line on the bottom)

[IanB]

bob.pointresize(width,height/2)

OMG. THAT is stunning easy! Why haven't I thought of it ;)Because you know what you are doing and instictively reach for the zero cost filters first.

separatefield.selectodd is all done with pointer shuffling, i.e. the GetFrame code just passes the same VFB around with mutated descriptors.

pointresize(width, height/2) takes a trip thru the resizer core code. i.e. it really moves bytes around in memory.

[525/60]

Sorry I have been gone a couple days. I have since realized that encoding does not completely smooth it out. It still looks similiar when you step through the avi, but it's not 'flicker' if you can't see it anymore when you play the file.

One or other of the fields (it's not clear which) will pass through unprocessed I think it is clear that we have been discussing simply passing through the top field.

I think it's worth pointing out too that the apparent sharpness of the untouched field is down to aliasing You're right. When I first wrote it, the idea was to minimize filtering, both for speed and to try to maintain the integrity of the original. Now I see that the minimum amount of filtering that needs to be done to maintain integrity while maximizing sharpness is to copy each line of the field and interpolate between fields. I still think the next step should be to temporally filter between the interpolated and non-interpolated lines.

Because you know what you are doing and instictively reach for the zero cost filters first.

separatefield.selectodd is all done with pointer shuffling, i.e. the GetFrame code just passes the same VFB around with mutated descriptors.

pointresize(width, height/2) takes a trip thru the resizer core code. i.e. it really moves bytes around in memory. That's cool, so the original is better. The code can be adapted to alternate between all the corresponding original and interpolated lines:

bob(0,1).separatefields().selectevery(4,0,2,3,1)
Try to smooth between each pair of fields
or sharpen the interpolated field using the original.
selectevery(4,0,3,1,2).weave


Stephen