Evil DV, need to deinterlace and stream

[Blue_MiSfit]

Hey folks,

So I've got a very nasty little compression project bearing down on me today.

Here (http://www.megaupload.com/?d=T46A98O2)is a sample. Sorry for the large size. DV is.. well... large :)

The source is a QuickTime DV MOV, which was edited in Final Cut. I can bring it into AviSynth with QTSource, and de-interlace without difficulty.

However, when I actually try to compress, the source is perhaps the most UNcompressible source I've ever looked at.

Upon further inspection, I think the interlacing structure is messed up, perhaps as a result of going from an HDV source to a DV timeline, and throwing in some chroma key material, and lower thirds stuff. Maybe you guys can help me out.

Just a simple AssumeBFF().Bob() does exactly what you WOULDN't think. The video bobs up and down, which is what bob is supposed to counteract. This was my first clue that something screwy was going on.

The only way I've been able to even come close to acceptable compression is by using AssumeBFF().SeparateFields().SelectEven().HorizontalReduceBy2() followed by fft3dgpu(sigma=2). Even so, a low bitrate 512kbit streamable XviD encode looks pretty bad, with an average quant of 6.9 with no bvops (quicktime compatibility). I would like to eventually generate 3 streams, one full resolution at ~768 with 64 for audio, one at a medium resolution at ~512 with 64 for audio, and a third at half resolution, at ~384 with 32 for audio (mono).

I think this video was shot in HDV (TFF interlaced), and thrown into a BFF DV timeline. But that's just sort of a semi-educated guess.

Any help would really be appreciated!

Thanks
~MiSfit

[foxyshadis]

No, Bob is called bob exactly because the video bobs up and down. =p The only protection it has is shifting the field toward the center, but horizontal edges and fine detail will still jump. You must use a smart bobber (yadif, tdeint, etc) to mostly counteract that effect, or selecteven like you did, if you don't mind the aliasing. (Or mcbob if you don't mind your encode time trending toward infinity...)

Bitrates that low, especially without b-frames, might need very aggressive detail-killers, like Deen and DNR, perhaps paired with fft3d. One way or another you'll have to sacrifice a lot to fit it all.

[Blue_MiSfit]

I stand corrected :) I guess I'm spoiled using a smart bobber all the time.. Just out of curiousity, how is a bob() any different from separatefields().selecteven().BilinearResizeBackToFullHeight()? I'm guessing it's what you stated: "shifting the field toward the center". Perhaps you could explain this in a bit more detail?

I have gotten much better results now using tdeint(mode=2) followed by aggressive fft3dgpu. [edit] aw hell, I'm just going to MVBob the whole stupid thing to a YV12 Lagarith, and go from there...

I was incorrect in assuming the original source, it was actually standard NTSC DV from a DVX100.

I think I can squeeze a 640x480 version of it into under 1.5mbit (minus 20% just in case, and 64k for audio), but we'll see... It's gotta stream. On that note, I've got to mess about with VBV in Xvid. I've had good luck with x264, but this has to be MPEG-4 SP, due to client-side CPU power limitations. Plus, it's going to be decoded by QuickTime, and on Windows we all know what that means.... :(

Thanks for smacking me upside the head though foxyshadis.. I need that :D

~MiSfit

[davidhorman]

how is a bob() any different from separatefields().selecteven().BilinearResizeBackToFullHeight()?

The latter will return only the even frames. If you take out the selecteven, you'll see that the frames really do bob up and down - this is because you've separated the fields, placing top and bottom fields right next to other. This effectively "loses" the information that the bottom field lines are meant to be one pixel below the top field lines. When you resize back up, the original bottom field lines are on the same lines as the top field lines (albeit in consecutive frames).

Your sample looks okay to me - try a nicer deinterlacer like yadif to improve the situation - it should help on the flickering text.

David

[foxyshadis]

You can still do x264 with quicktime. Start with the MeGUI QT profile, and just lower things until it works on the PC - b-frames are probably out, inloop or cabac or both might be out, but at that point it should be as fast as SP with (hopefully?) better quality. At least it's worth a try.

Bob is internally just SeparateFields() and BicubicResize(width,height*2,b,c,0,-.5,0,-.5) (bottom) and +.5 (top) alternating. (I'm not going to type up the actual script, you know what I mean.) So it's not a true, bounce-up-and-down bob like eedibob or ibob, but the effect will still be pronounced in some areas. Bob(0,1) will keep original fields entirely intact, but increase the bobbing effect.

[davidhorman]

Bob(0,1) will keep original fields entirely intact, but increase the bobbing effect.

I'm a bit confused as to why bob() wouldn't do that anyway - in the docs it says about bob(0,1):

To strictly preserve the original fields and just fill in the missing lines.

So what is a plain bob() not doing, strictly speaking, to preserve the original fields?

David

[foxyshadis]

In normal Bob (1/3,1/3) the original field lies somewhere between every pair of output fields, so both of them are interpolated from it, and its neighbors. I was off a bit, it's only ±.25, not ±.5. Doing so increases temporal stability at the cost of spatial sharpness and some extra artifacting; if you raise the detail level you have to reduce stability, with such a simple algorithm, and vice versa. The defaults are a compromise.

If you Bob.Selecteven, temporal stability is moot and you may as well keep the field you had whole.

[scharfis_brain]

loadplugin("c:\x\eedi2.dll")
loadplugin("c:\x\tdeint.dll")
loadcplugin("c:\x\yadif.dll")

directshowsource("Big John Trailer chop.mov")
input=directshowsource("Big John Trailer chop.mov").converttoyuy2()
u=input.utoy().turnright().separatefields().selecteven().eedi2().turnleft().converttoyv12(interlaced=true).yadif(mode=1).converttoyuy2()
v=input.vtoy().turnright().separatefields().selecteven().eedi2().turnleft().converttoyv12(interlaced=true).yadif(mode=1).converttoyuy2()
ytouv(u,v,input.converttoyv12(interlaced=true).yadif(mode=1).converttoyuy2())This is the script to deinterlace it within reasonalbe time without much losses.
The script re-interpolates the 4:1:1 chroma to 4:2:2 thus reducing chroma stairstepping.
Furthermore it uses a trick to abuse yadif() to work with YUV 4:2:2 chroma.

For good copressibility I'll try some motion blurring tricks.

[davidhorman]

I was off a bit, it's only ±.25

When you say ±, do you mean in the same direction (which presumably results in slight blurring) or in different directions? If it's the latter, I don't get how moving the fields relative to each other would increase stability :confused:

David

[scharfis_brain]

+.25 up for the one field, -.25 down the other field in differential related to the height of a field.

So that means:
you could do a 0.5 pixels shift of one field, while leaving the other one alone. (not good, as one field get blurred, while the other one remains crisp)
Bob() does its work on full height frames, so it is a 1 pixel vertical shift. This also explains, why it allows you to leave the original fields untouched.

[scharfis_brain]

loadplugin("c:\x\eedi2.dll")
loadplugin("c:\x\tdeint.dll")
loadplugin("c:\x\mvtools.dll")
loadcplugin("c:\x\yadif.dll")



input=directshowsource("Big John Trailer chop.mov").converttoyuy2().crop(8,60,-8,-60)

u=input.utoy().turnright().separatefields().selecteven().eedi2().turnleft().converttoyv12(interlaced=true).yadif(mode=1).converttoyuy2()
v=input.vtoy().turnright().separatefields().selecteven().eedi2().turnleft().converttoyv12(interlaced=true).yadif(mode=1).converttoyuy2()

deint=ytouv(u,v,input.converttoyv12(interlaced=true).yadif(mode=1).converttoyuy2()).converttoyv12()
deint=deint.bicubicresize(640,360)


backward_vec = deint.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)

forward_vec = deint.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = deint.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = deint.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = deint.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
deint.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=FramerateNumerator(deint)*8,den=FramerateDenominator(deint), idx=1,idx2=2)
temporalsoften(7, 255,255, mode=2)
selectevery(16,6)

finally I found this script to work okay with 1000kbps xvid.
it blurs the motion to a severe extend, but this reduces the blockyness.

[davidhorman]

Bob() does its work on full height frames, so it is a 1 pixel vertical shift.

Oh, I see now, thanks Scharfi :thanks: :stupid:

David

[Didée]

Bob()'s action is a thing where I still have one certain problem with understanding:

+.25 up for the one field, -.25 down the other field in differential related to the height of a field.
[...]
Bob() does its work on full height frames, so it is a 1 pixel vertical shift. This also explains, why it allows you to leave the original fields untouched.

How can that be? Theoretically, it can not ...

If bob does shift the frames up/down by +-0.25 pixels, then it can not preserve the original fields: Shifted fields are no more original. They are shifted.

Is it so, perhaps, that the actual differences by 1/4-shifting are so small that the original fields are *almost* preserved, with most of the differences being lost in rounding of small numbers?

IIRC, in a test some time back I found that Bob()-ing a YV12 interlaced source showed no differences to the original luma fields; however it showed a mentionable difference on the chroma fields (due to the halved vertical resolution of YV12 chroma).

Someone help an interlacing impared gramps over the bobbing street, please? :D

[scharfis_brain]

okay, I'll roll it up from the root.

1) if you do a plain separatefields() you'll end with an up and down jumping video.
In relation to the height of the fields, this up and down movement has an amount of 1/2 pixel.

At the height of the fields there basically are two options to avoid this bobbing:

2a) shift the even fields up (or down depending on parity) by 1/2 pixel and leave the odd fields untouched.
This will result in a stable video but its even fields are blurred while the odd fields remain very crisp.

2b) to avoid this sharp-blur-sharp-blur alternation one averages the shifting process for field-height output:
the even fields become shifted 1/4th pixel upwards (or downwards depending on parity) and the odd fields become shifted 1/4th pixel downwards (or upwards depending on parity). this blurs the whole video a bit but has no temporal jitter regarding position as well as sharpness.

3) for full frame height, at first it seems obviously to up-interpolate both fields to the frame height and then shift them +1/2 pixel (even) and -1/2 pixel (odd) in order to avoid the vertical jitter (much like 2b).

But at the second look you'll notice that the ABSOLUTE shift between both is 1 pixel. So after upsizing the fields to frame height bob() effectively leaves the even fields alone while shifting the odd fields by one pixel.
This method is the same like 2a) but it is lossless and thus doesn't show any blurring.
(Of course bob() internally works more instantly. It doesn't divide the process into resizing and followed shifting. It's a single transform)

regarding interlaced YV12 and bob(): there must be a bug in bob()'s chroma sampling routines.

if I run bob(0,1) on each chroma plane individually, I get much crisper chroma output after reinterlacing. I also checked for chroma displacement but found none in my bob-wrapper:

i=last
y=i.bob(0,1)
u=i.utoy().bob(0,1)
v=i.vtoy().bob(0,1)
j=ytouv(u,v,y)

(This script also allows you to run YV12 only deinterlacers in YUY2)

[IanB]

IIRC, in a test some time back I found that Bob()-ing a YV12 interlaced source showed no differences to the original luma fields; however it showed a mentionable difference on the chroma fieldsAaahhhggg!!!

Yes it will due to the chroma location in interlaced YV12 fields. Damn!

The chroma sample position for top field chroma in a frame is between lines 0 and 1, i.e. {0.5, 4.5, 8.5, ...}.

The chroma sample position for bottom field chroma in a frame is between lines 2 and 3, i.e {2.5, 6.5, 10.5, ...}.

The resizer core input reference is that the first line is 0, the second is 1, etc. To this input reference we add the offset (+/-0.25) then multiply by the scale factor, 2 in this case, to create the output reference. The various resizers then use this formula to interpolate each output pixel relative to the input pixels. For BilinearResize the interpolation is simply the weighted average of the 2 bounding pixels. For the other resizers the interpolation is increasingly complicated based on the order of the filter and uses more surrounding pixels.

Thus without any input offset the output to input line mapping is thus :-output input
0 0.0
1 0.5
2 1.0
3 1.5
...Now this is fine for the luma channel, output line 0 is input line 0, and for the top field, output line 0.5 is input line 0.5, but for the bottom field, output line 0.5 is input line 2.5, oops!


P.S. In testing there is an effect as described above but it effecting both fields equally but opposite not just the bottom field as expected.

And the Bob(0, 1) trick also does not work as expected, there is something else in play here.

[Blue_MiSfit]

:O wow guys, this has turned into such an academic discussion! I Feel special :)

Thanks for all your throughts! I will try out Scharfi's script..

I just MVBobbed the whole stupid thing over night (4 minutes took over 2 hours :D) into lagarith, but I still need to de-noise it, and good 4:1:1 to 4:2:2 interpolation is something I've always wanted to master. I wish we had done that before chroma keying.. Pretty terrible results there..

Thanks again!!!

A note, strangely when I try to use DirectShowSource to open the QuickTime MOV, AviSynth throws "the scripts return value is not a clip", with just the line DirectShowSource("I:\Big John.mov"). QTInput works fine, but always returns YUY2. I'm assuming this is ok, because the result of the whole script looks better than the input.

If only I could easily remux to an AVI and use AviSource to have Cedocida return 4:1:1, but alas doing so while maintaining seeking has so far eluded me..

~MiSfit

[IanB]

// the following translates such that the image center remains fixed
...
pos = subrange_start + ((subrange_width - target_width) / (target_width*2));This probably has a lot to do with this issue. For a times 2 upsize it add a -0.25 input pixel shift.

[Blue_MiSfit]

Thanks again guys. I got great results using Scharfis_Brain's script, and was able to compress at 640x360 at just over 1mbit. Mission accomplished!

~MiSfit