HD 1080i to DVD (all interlaced) - how?

[Blue_MiSfit]

A very simple example:


mm=tmm()
TDeint(mode=1, emtn=mm)


~MiSfit

[PatchWorKs]

For those who don't know, MediaCoder build 4130 (http://mediacoder.sourceforge.net/wiki/index.php?title=MediaCoder_Changelog#0.6.1.4130) now has the ability to correctly decode AVCHD (the format used by many HD camcorder)

Enjoy !

[czerro]

I'm curious, what is the incentive to Bob vs IVTC? I understand it makes it easier to convert back to 29.97fps with weave() thus making it easier to author, but there are tricks to get around that restriction (ie fake the framerate).

I have never authored DVD's but dealt with SVCD encoding back in the early days of portable image backup when DVD authoring at 2x for a 200 dollar drive (20 dollar 1-time DVD) was not really an efficient or cost-effective option... And many intelligent people, knowing how much 23.976 vs 29.97 fps can effect image quality on an 800 meg disc mpeg-2 *Hardware Player Compatible* stream, created programs to fake 29.97 flags on 23.976 sources. DVD hardware WILL play 23.976 source with 23.976 flags, but with a noticeable intermittent stutter. Also audio would often get out of sync on FastForward/Rewind/Pause/etc. DGPulldown was a genius program that worked by magic and dark voodoo (I'm pretty sure anyway) and would somehow by simply changing the flags in your 23.976 source (30 seconds of work) convince any hardware or software player that it was rendering a 29.97 fps stream. Intermittent stutter gone, and looked better than a "true" 29.97 fps "fake" (progressive as opposed to interlaced) stream.

I assume Avisynth's ChangeFPS(), AssumeFPS(), options perform some sort of similar function, although I've never had any reason to deal with them. Why is IVTC flag tampering not a superior way to maintain bits per frame (quality) over Bob/SelectEvery/Weave? I'd rather get away with compressing/rendering an HD source at 23.976 fps than 29.976 fps considering the framesize (1080i!)

[Blue_MiSfit]

I'm curious, what is the incentive to Bob vs IVTC?


The two are to be used for completely different sources.

In both cases, we want to take interlaced video, and deinterlace it without loosing info. However, the two types of sources (hard telecined video and pure interlaced videos) need to be handled differently.

Telecined video is essentially a progressive stream that has undergone a (sometimes nasty) process to make it "fit" inside a 29.97fps interlaced stream (assuming NTSC). So, to make 24p film content fit inside 60i ntsc, you cut the 24p into 48 fields per second, then duplicate some fields here and there, until you reach 60 fields per second. That's how telecine works.

If it's done properly, telecine is usually pretty simple to remove. If it's a common 3:2 pulldown, a simple TFM.TDecimate will remove the extra fields, and re-assemble the original 24p. There are more exotic methods for more exotic / broken sources, but this works most of the time.

Now, bob-deinterlacing should typically only be applied to TRULY interlaced video. i.e. when you do SeparateFields, and step through, each field represents a distinct temporal location. This is opposed to telecined material (where you have some duplicated fields that can just look a tiny bit spatially different).

Bobbing will take each temporally distinct field, and interpolate it to a full frame, using any number of special methods to use detail from other fields to assist in the interpolation process.

So, bobbing returns 60p. Same-rate deinterlacing can be as simple as bobbing followed by SelectEven, or there can be a dedicated same-rate algortihm. Same-rate deinterlacing returns 30p

(note, that 30p=29.97fps progressive, 60p=59.94fps progressive, 24p=23.976fps progressive, 60i=29.97fps interlaced).

The two are distinct procedures that should not be confused. One cannot simply apply IVTC to a pure interlaced stream, and expect meaningful results.

~MiSfit

[florinandrei]

TempGaussMC could be a choice for a "not-too-slow" mo-comp'ed bobbing filter, when used with fast settings ( (1,1,0,"Yadif"), essentially).

I tried to call the filter like this: TempGaussMC(1,1,0,"Yadif") and it gave me an error.
I tried it like this: TempGaussMC(EdiMode="Yadif") and it's slower than MCBob().

What's the correct syntax for TempGaussMC() that makes it faster than MCBob() without losing too much quality?

[Didée]

I tried to call the filter like this: TempGaussMC(1,1,0,"Yadif") and it gave me an error.
I tried it like this: TempGaussMC(EdiMode="Yadif") and it's slower than MCBob().
Since you managed to use named parameter syntax in your 2nd line (EdiMode="Yadif"), I think you'll be able to figure how to use the same named syntax for the 1st line.

Testing with SD content, everything with default settings (i.e. empty parantheses: MCBob() / TempGaussMC_beta1() ), I found that TempGaussMC is 40% faster than MCBob. With really fast settings, it's running 4 times the speed of MCBob (though the results are not all that brilliant, then). Moreover, TempGaussMC has a lower memory usage than MCBob. In short, with SD I cannot replicate your findings.

Figuring the special needs of HD-1080 is the job of you guys. If you want me to figure those needs, no problem: feel free to provide me with the according hardware ... a beefy quadcore with plenty of RAM and a mighty GPU will do for now. :)
(Since ingenuity doesn't automagically reflect in $$$ or €€€, it's beyond my possibilites, sorry. Please excuse that I am poor.)


NB, you're really shooting the butterfly on a wheel. Going 1080i -> 480i means your going from 100% image area down to 16.6% image area, or the other way round: you're loosing 83.3% image area in the process. Still, you want to use golden nails to hammer some planks together? The vast majority of the intermediate improvement (at the 1080p stage) will be lost again when you've reached 480i. A plain and fast bob() does pretty good for that task ...

[2Bdecided]

I think he's just a masochist. At least two of us have told him bob() is good enough!

btw, for typical camcorder footage shot without a tripod (i.e. the camera is never really still, so fields never really match) yadiff becomes the same as bob(), doesn't it?

Cheers,
David.

[florinandrei]

OK, I get it. :)

I'm focused on Yadif() since it's fast enough - I get between 5 and 6 fps from it, which is good enough: this way it could make a DVD before I get back from work if I start it in the morning.
The better bobs are too slow.
Also, I can't distinguish visually between Yadif() and the slower bobs - not for this kind of processing anyway.

But now I have another issue. I just made a DVD-RW with 3 different algorithms - separatefields/shift/interleave, tdeint/mmc, and yadif - and tested it on the PS3 with an HD (progressive) screen.
Shift looks okay, more or less.
But the other two vibrate very badly when there's motion, as if the order of fields is mismatched, or something. The script is:

SetMTMode(2,2)
LoadCPlugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll")
DirectShowSource("00000.m2ts", audio=false)
Yadif(mode=1)
LanczosResize(720,480)
AssumeTFF()
SeparateFields()
SelectEvery(4,0,3)
Weave()

The HCenc ini is:

*MAXBITRATE 9500
*PROFILE best
*AUTOGOP 15
*CQ_MAXBITRATE 5.000
*AQ 1
*DC_PREC 10
*INTERLACED
*TFF
*NOSCD
*MATRIX mpeg
*LUMGAIN 1
*PRIORITY normal
*WAIT 1

This is the file:

http://dl.getdropbox.com/u/29966/00000-yadif.m2v

I think I'll start by sprinkling AssumeTFF() in various places and watch the results. Maybe it's because I'm forcing TFF in HC.ini and it's not TFF? But AssumeTFF() in the .avs should take care of that. :confused:

It's kind of surprising to fail like this. :mad: IIRC, I tested it before and it was fine. But the scripts have changed quite a bit meanwhile.

[Blue_MiSfit]

What does this look like:


MPEG2Source(...)

AssumeTFF()
SeparateFields


Step through it frame by frame. Is the movement smooth or jittery? If it's jittery, try AssumeBFF.

Don't sprinkle randomly :) Determine the field order, set it, and forget it :)

~MiSfit

[Didée]

DirectShowSource does not hand over the source's actual field order to Avisynth. If Avisynth doesn't have a discrete filed order info, it defaults to BFF. Yadif picks up the field order provided by Avisynth. Since your source is TFF, that's where things go wrong.

Possibility #1:DirectShowSource("00000.m2ts", audio=false)
Yadif(order=1,mode=1)

Possibility #2: (mostly preferred) DirectShowSource("00000.m2ts", audio=false)
AssumeTFF()
Yadif(mode=1)


Determine the field order, set it, and forget it
The "forget it" part can be dangerous, in case your filterchain contains a filter that always resets field order to BFF, such as Bob(). ;)

[florinandrei]

I just came back from the living room after watching the new version when I saw the new replies. :)

Yes, I fixed it with Yadif(mode=1, order=1).
I'll try AssumeTFF() before Yadif, although I'm not sure what's the difference. Is it because AssumeTFF() is applied to everything afterwards, not just to Yadif()?

On the big progressive screen, the power lines hanging in the air are a bit jagged. This could be just the limit of the deinterlacer on the Playstation3, which is usually very good but I assume is not perfect.

I'll try it on the native interlaced CRT - that should be the "make it or break it" test.

[Gavino]

I fixed it with Yadif(mode=1, order=1).
I'll try AssumeTFF() before Yadif, although I'm not sure what's the difference. Is it because AssumeTFF() is applied to everything afterwards, not just to Yadif()?

The two have the same effect, but it's best to get into the habit of putting Assume[T/B]FF straight after DirectShowSource. Then the rest of the script has the right information no matter what you put in it (except for broken Bob() :)).

See it as part of setting up your source, rather than as part of deinterlacing.

[Alex_ander]

Yes, I fixed it with Yadif(mode=1, order=1).
I'll try AssumeTFF() before Yadif, although I'm not sure what's the difference. Is it because AssumeTFF() is applied to everything afterwards, not just to Yadif()?

AssumeTFF() before Yadif will be ignored (unless you use order=-1), since it is taken from declared 'order' parameter (in case you use wrong 'order' value, you'll at best get field order reversed on output). After Yadif you get progressive video for which it doesn't matter. It is only important to define field order before SeparateFields(), otherwise AviSynth default BFF will be used at separation thus inverting field order (from expected) for interlaced output.

[Gavino]

AssumeTFF() before Yadif will be ignored (unless you use order=-1)
... which is the default.
It is only important to define field order before SeparateFields() ...
This seems to contradict what you've just said, as deinterlacing filters (like Yadif) will also generally use the assumed field order by default.

So the cleanest way is always to set it after a source filter that doesn't get it right, like DirectShowSource.

[Alex_ander]

... which is the default.

This seems to contradict what you've just said, as deinterlacing filters (like Yadif) will also generally use the assumed field order by default.


I don't see any contradiction here. 'Default' for order=-1 in Yadif means the field order assumed by AviSynth just before applying the filter and it can be set directly with AssumeTFF(). And this setting is used only for this operation and doesn't apply to the newly created progressive video. By default, for any field operation with it BFF setting will be used by AviSynth. For this reason it is important to set the wanted field order just before field separation.


So the cleanest way is always to set it after a source filter that doesn't get it right, like DirectShowSource.

This would work the same as setting field order for bob-deinterlacing, but only in case order=-1 for Yadif. Internally setting order=0 or 1 in Yadif or LeakKernelBob overrides any field order setting before them.

[Gavino]

Where I saw a contradiction was: you seemed to imply AssumeTFF has no effect except on SeparateFields, immediately after stating that it does have an effect on the behaviour of Yadif(order=-1). But that wasn't my main point.

And this setting [AssumeTFF] is used only for this operation and doesn't apply to the newly created progressive video. By default, for any field operation with it BFF setting will be used by AviSynth.

That's wrong. Try this:

DirectShowSource(...)
AssumeTFF()
Yadif(mode=1) # or any other mode
Info()

You will see that the TFF property is preserved.

Any filter that doesn't do this (except for something explicitly designed to reverse field dominance) is broken.
Unfortunately Bob() falls into this category. :(

But it's true that LeakKernelBob/Deint requires you to explicitly specify the order, so AssumeXFF doesn't do you any good with that filter anyway.

[Alex_ander]

That's wrong. Try this:

DirectShowSource(...)
AssumeTFF()
Yadif(mode=1) # or any other mode
Info()

You will see that the TFF property is preserved.


I've just tried this:
LoadPlugin("c:\Program Files\AviSynth 2.5\SpecialPlugins\LeakKernelDeint.dll")
DirectShowSource("E:\Transfers\Video\UHeep\01 - Lady In Black [ r0xb0x ].mpg")
AssumeTFF()
LeakKernelBob(order=1)
info()
and I clearly see BFF as field order. I did it before and that's why I expressed such conclusion. I'll test it for Yadif now to see the difference.

[Alex_ander]

And yes, it is different with Yadif: 'Assumed TFF' and it is taken from the previous to Yadif line. Any idea, why LeakKernelBob behaves differently?!

[Guest]

Any idea, why LeakKernelBob behaves differently?! Differently than what?

[Gavino]

And yes, it is different with Yadif: 'Assumed TFF' and it is taken from the previous to Yadif line. Any idea, why LeakKernelBob behaves differently?!
Looks to me as if LeakKernelBob inherits the bug exhibited by plain Bob. Just checked this and LeakKernelDeInt does it right.

Looking at the source, I see that both Bob and LeakKernelBob invoke AssumeFrameBased (http://avisynth.org/mediawiki/AssumeFrameBased) on their result. This in turn resets the clip properties to the default (BFF). I now find myself confused as to whether the bug is in Bob/LeakKernelBob, or in AssumeFrameBased itself, or perhaps there is no bug at all and that's how it's supposed to work. :( :confused:

What's clear is that Yadif behaves differently from Bob/LeakKernelBob in this respect.

[IanB]

The Bob()/AssumeFrameBased() parity reset issue is to be addressed in 2.6

:Edit: The current behaviour is to reset the parity state to default. This is not wrong, it is just the way it works.

The revised behaviour will be to copy the parity from frame 0 of the input clip, this is the behaviour that most of the external Bob filter authors have adopted, and what those who have discussed and argued about the issue believe is a more expected, practical and manageable behaviour.

The change is not transparent so scripts that rely on the current behaviour may need to be adjusted, this is why the change has been deferred until the next major revision.

[Gavino]

The Bob()/AssumeFrameBased() parity reset issue is to be addressed in 2.6
Ian, just to put our minds at rest, can you clarify how it should work and which filters are currently 'wrong'?

I thought I understood it but now I'm not so sure...

[Alex_ander]

BTW, TDeint in bob mode keeps the previous field order (like Yadif).

[Alex_ander]

The current behaviour is to reset the parity state to default. This is not wrong, it is just the way it works.


I've always thought it's OK (and even more natural) to have field order reset to default just after creating different type of video by bob, for the following reasons.
1.The fields which were top/bottom before bob no more belong to the frames for which field order was determined.
2.Most bob filters use additional parameters (=assumed field order) to output frames in correct order and thus ignore input field order used by AviSynth.
3.In case the video is further used as progressive, field order doesn't matter (for encoding, etc.), also in case the video is further re-interlaced, it is easier to force AssumeXFF(), than to first examine which order is assumed by a particular bob filter. This gives predictable field order for interlaced output. I've always done so and yesterday it was a discovery to me (since it never influenced on results) that some of the bob deintelacers I use bring input field order to their output, despite the fact they don't use it for their internal work (except when they are 'told' to do so).

[2Bdecided]

I totally disagree. The field order is a property of the video clip, and AVIsynth knows this.

Any bob which is just a bob is faulty by design if you have to tell it the field order, when it's already known within AVIsynth.

If you are going to use the resulting progressive video as-is, the nominal field order is irrelevant - so preserving or changing it is just fine.

If you are going to re-interlace the video, then you usually know if you want to change the field order - otherwise the natural and obvious thing to do is to maintain the original field order. Breaking this behaviour is a bug IMO, not a feature.

Cheers,
David.

[Alex_ander]

The field order is a property of the video clip, and AVIsynth knows this.
Field order is property of interlaced clip (apart from AviSynth) and keeps information where different parts of the frame should be displayed on time line. AviSynth keeps field order value for every clip but with progressive clip you are absolutely free to invert it for any further operation.


Any bob which is just a bob is faulty by design if you have to tell it the field order, when it's already known within AVIsynth.
Which one is not faulty then?
One of the older and respected ones LeakKernelBob() only takes it as internal parameter (default is 'none' by manual) and that's good: you'll never forget to determine actual field order for your imported clip, which not always coincides with what AviSynth gets to 'know'.

If you are going to re-interlace the video, then you usually know if you want to change the field order - otherwise the natural and obvious thing to do is to maintain the original field order. Breaking this behaviour is a bug IMO, not a feature.

To 'change field order' of progressive clip (the one it's labelled with)? No. At re-enterlacing you decide, which fields in which order you want to take from a double framerate progressive video, where frames go in correct order (due to correctly set bob filter). Then half of them you throw away and get interlaced video with field order you want, not the one inherited. If you Assume the wanted field order before field separation, you'll get it without taking on belief some 'original' order you want to change or leave as is.

[IanB]

@Alex_ander, 2Bdecided,

Your 2 posts are the 2 summation arguments from a very long and drawn out development thread. :search:

There is no right answer. It came down to what was a more expected, practical and manageable behaviour. For me the pivotal argument was, that it is not trivial to copy the parity across the Bob() filter script wise, but it is easy for the filter to internally copy the input parity to the output clip. And it is easy script wise to ignore this action and keep the default behaviour.

[2Bdecided]

IanB, that makes sense.

However, I have my own, less convincing argument...

AVIsource("clip.avi")
somebob
separatefields().selectevery(4,0,3).weave()
...should be a NOP, or as close to one as possible. (a) because you'd expect it to be, and (b) because a common use for this is to perform progressive only filtering on interlaced content after the bob. There's no sense forcing the script to be more complicated than it needs to be.

There is no right answerFor a developer, you're too humble. Or maybe not. ;)

Cheers,
David.

[IanB]

@2Bdecided,... that it is not trivial to copy the parity across the Bob() filter script wise ...As an exercise try to code up a script that behaves as your example does with the current Bob() rules.

[Gavino]

However, I have my own, less convincing argument...

Sounds pretty convincing to me. :)
There's no sense forcing the script to be more complicated than it needs to be.
Exactly. You're free to sprinkle AssumeXFF calls throughout your script just to be on the safe side (and given the current state, that's maybe not a bad idea), but in the absence of such calls, it seems logical to preserve the setting across filters.

[Didée]

@2Bdecided,As an exercise try to code up a script that behaves as your example does with the current Bob() rules.

I'm not quite sure if I get you ATM ... are you heading for the following almost-trivial wrapping code for Bob() ?
(BTW, yes the wrapper is missing the b,c parameters)

# wrapping bob(), this means bicubic interpolation of fields
# this function also passed the Fieldorder information to the output,
# unlike bob() which always resetts the fieldorder to BFF.
# The current field stays untouched.

Function DumbBob(clip Input, int "Height")
{
Input.Bob(0, 0.5, height = Height)
GetParity(Input) ? AssumeTFF() : AssumeBFF()
}

[Leak]

There is no right answer. It came down to what was a more expected, practical and manageable behaviour. For me the pivotal argument was, that it is not trivial to copy the parity across the Bob() filter script wise, but it is easy for the filter to internally copy the input parity to the output clip. And it is easy script wise to ignore this action and keep the default behaviour.
:D

But it's also perfectly possible to create a wrapper around LeakKernelBob that will feed it the order based on GetParity and use the correct AssumeXFF afterwards if you want that behaviour...

EDIT: Ugh, didn't notice there was another page where Didée already did (almost) exactly that... ^_^;;;

np: Boy Robot - Magic Toys For Girls And Boys (Rotten Cocktails)

[2Bdecided]

But that's a horrible bodge to fix an obvious bug. Much better to fix the bug itself.

...then if anyone still wants or needs to sprinkle their scripts with assumeXff, they can still do so.

...and those who don't need to think about field order, don't need to.

Cheers,
David.

[florinandrei]

Yadif(mode=1, order=1).

On the big progressive screen, the power lines hanging in the air are a bit jagged. This could be just the limit of the deinterlacer on the Playstation3, which is usually very good but I assume is not perfect.

I'll try it on the native interlaced CRT - that should be the "make it or break it" test.

I just did that. The power lines are still a bit jagged. Maybe I forgot how the interlaced display is supposed to look like. But I remember that interlaced video on interlaced display is not supposed to look quite that jagged.

Here's what: tomorrow I'll go back to the same place. I'll take the HD camera, and also the old DV camera, and shoot the same scene with both. Then I'll convert the HD footage to SD and compare the two on an interlaced display. IMO, there should be no difference between the two, other than that the sequence originally made on HD should look better (better camera, then it's scaled down).

I'll let y'all know what happens.

[florinandrei]

EDIT: There was no "shimmering". Bad choice of words. It was aliasing. I'll reply below and explain.

I went back to the place with the power lines, two cameras in hand: HD and SD (DV Super8). Both cameras shoot 16:9. The SD camera is pretty cheap and amateurish.
The SD video was encoded to DVD/MPEG2 with no processing (other than ConvertToYV12(interlaced=true)). The HD video was converted to DVD by 3 methods: Bob()/Lanczos(), separatefields/shift/interleave, and Yadif()/Lanczos().
I made a DVD with all 4 clips and compared them on a regular 4:3 SD (480i) TV and a plain old DVD player connected via Component.

Both the TV and the DVD player have a "16:9" mode; I guess the TV in 16:9 will just scan a smaller area on the screen so as to match the 16:9 aspect ratio, while the DVD player in the 16:9 mode will simply fill the whole screen with video when playing a 16:9 DVD (will not letterbox).
So I have to either set both the TV and the DVD player to 4:3, or both to 16:9. The result is roughly the same, but in 16:9 mode a 16:9 DVD will look sharper (letterboxing is wasting scan lines, while in 16:9 there's no letterboxing).

Anyway, the results with the TV and DVD player in 16:9 mode:
Bob() and Yadif() look very similar. I can't really tell, visually, if there's a big difference (and Bob doesn't encode much faster than Yadif either). Thin horizontal lines (power lines hanging in the air) are shimmering a little bit when the camera moves vertically. If they are not perfectly horizontal, they also look a bit jagged.
Shift/interleave has zero shimmering/jaggies. However, I have a hard time telling whether it's more "soft" vertically than Bob(). I know on the computer screen shift/interleave looks soft. On the interlaced 480i "16:9" TV I don't think I can actually see a difference. What I do see might be due to my imagination. This was very surprising.
The native SD video looks more similar to shift/interleave than to Bob(). It has essentially zero shimmering/jaggies - well, it does have a very tiny amount, which might be due to the 4:1:1 color "softness" of the master format (DV), which produces nasty color shimmering on hard edges.

In 4:3 mode all videos shimmer a bit at the power lines. This is expected, since it maps a 480i video on a "300i" screen, or something like that.

Compared to what I see on the computer screen, the verdict after watching on the 480i TV is very surprising. Shift/interleave is the most similar to a full native 480i processing chain. It doesn't shimmer, just like native 480i, and unlike Bob or Yadif. And it's not softer vertically than Bob or Yadif - if there is any softness, I'll have to do a lot more tests to reveal it, and the vast majority of viewers will not likely notice it.

I would love to hear your comments.
What I don't quite understand is how shift/interleave looks so soft (vertically) on the computer as to seem almost deinterlaced :-) but on the native 480i screen it looks similar to the native 480i video. I know there's less resolution on the TV, but I'm pretty sure that's not the whole story.

Here are the MPEG2 files:

Bob:
http://dl.getdropbox.com/u/29966/hd2dvd/hd-bob.m2v
Shift:
http://dl.getdropbox.com/u/29966/hd2dvd/hd-shift.m2v
Yadif:
http://dl.getdropbox.com/u/29966/hd2dvd/hd-yadif.m2v
Standard def:
http://dl.getdropbox.com/u/29966/hd2dvd/sd.m2v

This is a typical bob .avs for converting HD to DVD:

DirectShowSource("hd.m2ts", audio=false)
AssumeTFF()
# Choose one of these
#Bob()
#Yadif(mode=1)
LanczosResize(720,480)
AssumeTFF()
SeparateFields()
SelectEvery(4,0,3)
Weave()

This is the shift/interleave .avs:

Global NewHeight=480
Global NewWidth=720

DirectShowSource("hd.m2ts", audio=false)
AssumeTFF()
SeparateFields()
Shift=(Height()/Float(NewHeight/2)-1.0)*0.25
Tf=SelectEven().LanczosResize(NewWidth, NewHeight/2, 0, -Shift, Width(), Height())
Bf=SelectOdd().LanczosResize(NewWidth, NewHeight/2, 0, Shift, Width(), Height())
Interleave(Tf, Bf)
Weave()

This is the .ini for HCenc for HD (the one for native SD is very similar). It's very high bitrate encoding, to maximize quality, but not quite CBR.

*MAXBITRATE 9500
*ASPECT 16:9
*PROFILE best
*AUTOGOP 18
*CQ_MAXBITRATE 5.000
*AQ 4
*DC_PREC 10
*INTERLACED
*NOSCD
*MATRIX fox1
*LUMGAIN 2
*PRIORITY normal
*WAIT 1

This is how I encode:

HCenc.exe -ini HC-hd.ini -i hd-video-yadif.avs -o hd-yadif.m2v -log hd-yadif.log -frames all

[henryho_hk]

florinandrei, :goodpost:

Could you post a sample of the original DV avi somewhere? :D

NTSC DV 4:1:1 should be "sharp" vertically (coz 4:1:1 dun have vertical chroma sampling). I tend to believe the softness of sep-resize-int method is due to the resizing of alternate field lines as if they were adjacent to each other.

But then I dun understand why bob/resize/selectevery are shimmering on TV. :confused:

[2Bdecided]

A few comments...

Firstly, it's interesting - the "separate fields shift corrected" resize does surprisingly well, but it's not performing magic. What it's doing is blurring the image in the vertical dimension, while the bob resize is keeping it sharp.

It you _want_ to blur the image in the vertical dimension to get the same results from bob, this is pretty close...

Bob()
assumetff()
LanczosResize(704,480)
blur(0.0,1.0)
separatefields().SelectEvery(4,0,3).Weave()

It reduces the lanczos halos a little compared to no bluring, which you may or may not want. (I think they look quite ugly on the "separate fields shift corrected" version, being generated at a 240-line resolution rather than 480-line resolution).



Secondly, in my testing (not related to this thread), sources which cause a problem (e.g. shallow sharp lines, repeating fine patterns) benefit from softening in the vertical dimension before interlacing; other source do not (and look beter sharp). Broadcast cameras and format converters typically use a compromise, softening a little, but not enough to remove all aliasing. The place to tune and set this is in the progressive domain (in this case, the output of bob) - by doing the "separate fields shift corrected" trick, you can't alter what you're doing so easily.



Thirdly, I'm still not convinced by that "Shift" value calculation - it fails for the trivial case - i.e. if the new height is the same as the source height, it still tries to shift the image, which can't be right.

I think it should be...
avcsource("00000.dga")
assumetff()

NewWidth=704
NewHeight=480
amount_to_crop=((float(last.height)/float(NewHeight))*0.5)-0.5

separatefields()

Tf=SelectEven().lanczosresize(NewWidth,NewHeight/2,0,0,0,-amount_to_crop)
Bf=SelectOdd().lanczosresize(NewWidth,NewHeight/2,0,amount_to_crop,0,0)

Interleave(Tf, Bf)
Weave()

This gives a shift of 0.625, rather than 0.875, for the current case. The difference is nearly invisible, but I think it's the correct calculation in the general case. I'm sure someone will correct me if I'm wrong. I recall seeing some kind of correction in a thread, but can't find it - I figured out the above myself from first principles.

Cheers,
David.

[Gavino]

I'm still not convinced by that "Shift" value calculation - it fails for the trivial case - i.e. if the new height is the same as the source height, it still tries to shift the image, which can't be right.

I think it should be...
...
amount_to_crop=((float(last.height)/float(NewHeight))*0.5)-0.5
separatefields()
Tf=SelectEven().lanczosresize(NewWidth,NewHeight/2,0,0,0,-amount_to_crop)
Bf=SelectOdd().lanczosresize(NewWidth,NewHeight/2,0,amount_to_crop,0,0)
I think you're on the right track (the trivial case proves the previous calculation was wrong), but I think it should be

amount_to_shift=((float(last.height)/float(NewHeight))*0.5)-0.5
# this is (total) required relative shift of odd v even in field-based co-ordinates
separatefields()
Tf=SelectEven().lanczosresize(NewWidth,NewHeight/2)
Bf=SelectOdd().lanczosresize(NewWidth,NewHeight/2, 0, amount_to_shift, 0, height)

Two points to note here:
- since the top field is not displaced by SeperateFields or Weave, it seems to me all the shift should be given to the bottom field;
- your solution was scaling an image of the wrong height, so changing the scale factor as well as shifting.

[Didée]

No. IanB's initial code for interlaced resizing is correct. Period.

Look once more at it:
Global NewHeight=480
Global NewWidth=720

DirectShowSource("hd.m2ts", audio=false)
AssumeTFF()
SeparateFields()
Shift=(Height()/Float(NewHeight/2)-1.0)*0.25
Tf=SelectEven().LanczosResize(NewWidth, NewHeight/2, 0, -Shift, Width(), Height())
Bf=SelectOdd().LanczosResize(NewWidth, NewHeight/2, 0, Shift, Width(), Height())
Interleave(Tf, Bf)
Weave()


This code does NOT fail in the trivial case. When old_height == new_height, it calculates a shift of zero.

Note that in the "shift=..." line, "Height()" refers to the height of the field separated clip, which then is divided by the half of new_height. This division results to "1.0" in the trivial case. Proceed with 1.0 - 1.0 which equals zero. Zero times 0.25 is zero. Hence, the shift is zero for the trivial case.

BTW, it was me who "corrected" IanB in the shimmering bars thread, and I was wrong with that. I confused the shift correction needed for bobbing with the shift correction needed for interlaced resizing, which isn't the same. (For bobbing, you've to omit the "-1.0" from the shift correction term, subsequently resize to new_height/1, and omit the final weave().)

On this matter, IanB was correct from the very beginning.

[Gavino]

@Didée.

I took what 2bdecided said about the trivial case at face value, and didn't check back against the original, again working from first principles.

What I ended up with is basically the same as IanB's version, with the difference that I gave all the shift to the bottom fields instead of splitting it between top and bottom. Can you explain why this is wrong? (Note the distinction, as you pointed out, between this case and the usual bobbing one.)

EDIT: Is it something to do with the resizer being center-based rather than zero-based? I still don't quite see it though.

[2Bdecided]

Note that in the "shift=..." line, "Height()" refers to the height of the field separated clipMissed that, thank you.

Cheers,
David.

[2Bdecided]

My shift amount is equivalent to half a line in the final field, minus half a line in the original field (one line in the final frame, minus one line in the original frame).

Ian's shift amount is exactly half this.

Proof:
Ian's formula:
Shift=(Height()/Float(NewHeight/2)-1.0)*0.25
Before separate fields, height was double, hence this is equivalent to:
Shift = ((last.height/2)/(newheight/2) – 1.0) * 0.25
Shift = ((last.height)/(newheight) – 1.0) * 0.25
Shift = last.height/newheight*0.25 – 0.25

Which is exactly half of what I had:
amount_to_crop=((float(last.height)/float(NewHeight))*0.5)-0.5


I crop my value from the top of the bottom fields, and the bottom of the top fields. Of course this scales it - the distance from the first to last line in one field of the lower res picture is a shorter distance than from the first to last line in one field of the original higher res picture (think about it).

Ian crops his value from top of the bottom fields, and adds his value to the top of the top fields. This gives exactly the same inter-field offset as me at the top of the field, but an overall offset of 1/4 line per final field (1/2 line per final frame) relative to me at the top of the field. Ian is keeping 1/2 line per final field more picture data than me.


I don't state all this as fact. I state it as what I think at 6:03PM Monday evening - feel free to prove me wrong.

Cheers,
David.

[IanB]

EDIT: Is it something to do with the resizer being center-based rather than zero-based? I still don't quite see it though.resample_functions.cpp:234 // the following translates such that the image center remains fixed
double pos;
...
if (fir_filter_size == 1) // PointResize
pos = subrange_start;
else
pos = subrange_start + ((subrange_width - target_width) / (target_width*2));Yes the -ve shift of the top field is to undo the centre correction calculation inside the filter generator. And the +ve shift of the bottom field is the opposite. I discussed this with Wilbert in Cropping interlaced video = shimmering bars where I coined the term output uncrop.

One thing to remember here is we are going from a 1080i source to a 480i display, a 2.25 times reduction in height, there is so much excess information here it really won't matter how you convert it, as long as you do it correctly. In the smart Bob case it is more like a 4.5 times reduction.

Using dumb Bob followed by SeparateFields() is a way of cheating the Nyquist limit. Effectively you are doing a 540-(Bicubic)->480-(Point)->240 the Interlaced Resize method does 540-(Lanczos)->240 so the low pass filtering frequency is approximately double (point resizing does no anti-aliasing (low pass) filtering)

[florinandrei]

OK, first off, I made a bad choice of words. There was no "shimmering" on the horizontal lines with those algorithms. What I was seeing was aliasing, stair-stepping. Sorry about the confusion, English is not my native language.

I did more tests. This time I put the DVD in the PS3 and watched on the 1080p screen. Everything looked pretty similar to the native 480i screen actually.
Bob and Yadif suffer from aliasing on the almost-horizontal lines.
Shift/interleave is rock solid, zero aliasing, and I had a discussion with a friend who was watching with me - I said I think it's slightly more blurry on the vertical than the other clips, while he said it was my imagination. After watching repeatedly, I didn't know what to think anymore. :rolleyes:
The native 480i video from the DV source actually had a little bit of aliasing on the big screen (unlike on the native 480i screen in 16:9 mode, where it was flawless), I guess due to the PS3's deinterlacing software.

I watched again everything on the interlaced CRT screen in 16:9 mode.
Bob and Yadif exhibit visible and annoying aliasing.
Shift/interleave has zero aliasing. Again, the same discussion: is it more blurry or not? I don't know if it's really blurry or it's just what I remember from the computer screen (on the computer it's clearly a bit blurry on the vertical). Even in 4:3 mode it still looked pretty good, maybe a hint of aliasing, barely noticeable if you know where to look.
The DV-source native 480i clip looks great in 16:9 mode (all 480 scan lines from the DVD are mapped onto 480 scan lines on the TV, since the scan area on the CRT is compressed vertically). There's some moderate aliasing in 4:3 mode (because there are 480 scan lines from the DVD mapped onto fewer than 400 lines on the TV).

I'm starting to realize just how bad the DV camera is. I wish I did the test with a better camera. Yes, it's just 480i, but even by that standard it's pretty lame.

Because this DVD will be watched on interlaced standard-def CRTs, most of them in 4:3 mode I guess, I am strongly inclined at this moment to just use shift/interleave and forget about Bob, Yadif and all that stuff. Unless somebody comes up with something new to try. :script:

Could you post a sample of the original DV avi somewhere? :D

I'll go one step above that.

Here's the DV file:
https://dl.getdropbox.com/u/29966/hd2dvd/sd.avi

And here's the AVCHD file:
https://dl.getdropbox.com/u/29966/hd2dvd/hd.m2ts

Now y'all have all the master files, all the results, and the scripts I used to do the processing.

Funny how the HD file is about half the size of the standard-def file, yet it has a lot more information. :) The technology really moves forward I guess.

Firstly, it's interesting - the "separate fields shift corrected" resize does surprisingly well, but it's not performing magic. What it's doing is blurring the image in the vertical dimension, while the bob resize is keeping it sharp.

Almost too sharp. There's aliasing in the image after bob that was not there before. I mean, compared to the native full end-to-end 480i chain, which looks perfect, bob clearly does something to the image, either adds some info, or removes something, or changes it, or a combination of the above.

Is it possible that the bob procedure is technically equivalent to an artificial sharpening?

Finally, my eyes tell me that bob needs "anti-aliasing". Isn't that essentially blur/softening? Are there ways to remove alias without making the image too soft?
Of course, this is all interlaced, which makes it pretty difficult to do anything. This forum needs a new icon - "I hate interlacing".

It you _want_ to blur the image in the vertical dimension to get the same results from bob, this is pretty close...

Bob()
assumetff()
LanczosResize(704,480)
blur(0.0,1.0)
separatefields().SelectEvery(4,0,3).Weave()

I'll try that, but honestly bob looks great on the computer screen. It's very sharp. If I could keep that and remove the jaggies... :)

And I'd rather use the other algorithm, if the result is the same, since it's faster. Or do you mean to tweak the blur to get an intermediate result?

It reduces the lanczos halos a little compared to no bluring, which you may or may not want. (I think they look quite ugly on the "separate fields shift corrected" version, being generated at a 240-line resolution rather than 480-line resolution).

What halos?

Secondly, in my testing (not related to this thread), sources which cause a problem (e.g. shallow sharp lines, repeating fine patterns) benefit from softening in the vertical dimension before interlacing; other source do not (and look beter sharp). Broadcast cameras and format converters typically use a compromise, softening a little, but not enough to remove all aliasing. The place to tune and set this is in the progressive domain (in this case, the output of bob) - by doing the "separate fields shift corrected" trick, you can't alter what you're doing so easily.

So, you're saying "do bob with softening while in progressive".

One thing to remember here is we are going from a 1080i source to a 480i display, a 2.25 times reduction in height, there is so much excess information here it really won't matter how you convert it, as long as you do it correctly.

That's what I expected. Yet the differences are fairly visible between the various scripts.

Using dumb Bob followed by SeparateFields() is a way of cheating the Nyquist limit. Effectively you are doing a 540-(Bicubic)->480-(Point)->240 the IntResize method does 540-(Lanczos)->240 so the high pass filtering is approximately double.

OK, that was a bit more than I could parse in one gulp. Can somebody rephrase this please?

[2Bdecided]

OK, that was a bit more than I could parse in one gulp. Can somebody rephrase this please?That bit is what you're saying by "bob is too sharp", and I'm saying by "some people filter vertically before interlacing".

1080i-bob-then-resize is giving you 480 sharp lines which are perfect for weaving back together, but almost "too" sharp for interlaced display, and far too sharp for just looking at 240 of them (aliasing - higher frequencies than can be accurately represented).

The separated fields resizing is only giving you about 240-320 lines of resolution in the 480 output lines, but that's probably enough for interlaced display.

The problem/trade off is that interlaced display isn't just 240 out of 480, nor is it just all 480 perfectly weaved back together - it's a mish mash of the two depending on what your eyes (or the deinterlacer, on a progressive display) make of it.


Halos are, for example, white parts around the edge of black objects - you can see them above and below the power lines.

Cheers,
David.

[henryho_hk]

VCD used to be fairly good on CRT TV and they look terrible on computer CRT monitor. Now DVD looks bad on a 22" LCD monitor. Monitors are really engineered to reveal the bad side of images.

BTW, what if we use BiCublinResize (http://ziquash.chez-alice.fr/) or even BilinearResize for the vertical dimension after Bob()?

[florinandrei]

BTW, what if we use BiCublinResize (http://ziquash.chez-alice.fr/) or even BilinearResize for the vertical dimension after Bob()?

I was kind of thinking along the same lines. Use a dumb resize instead of a smart one. But perhaps that would be equivalent (to some extent) to Lanczos and then blur?

I'll have to try this.

[florinandrei]

I took the Yadif script and made some variations.

First off, I tried 3 different resizes: Bilinear, Lanczos, and Spline36.
Lanczos and Spline36 are very hard to distinguish. The Yadif/Lanczos script is exactly the one I posted above, and I commented it already (crisp but aliased).
Bilinear seems to be less crisp, aliasing is not quite so bad but still visible.

Then I did a Yadif/Lanczos, either plain, or followed by blur(0.0, x) where x = (0.25, 0.50, 0.75, 1.00)
Obviously, increasing the x makes the video more blurry and reduces aliasing. With 0.75 aliasing is almost completely gone and it's a bit blurry but not too bad. Even with x=1.0, there is still a very tiny amount of aliasing visible. It's not exactly the same like shift/interleave.

I guess I will have to compare shift/interleave with blur with x either 0.75 or 1.0.
But I can't do that right now, I think I just bricked my main computer with a BIOS update. :mad:

[2Bdecided]

Even with x=1.0, there is still a very tiny amount of aliasing visible. It's not exactly the same like shift/interleave.I thought that too. If you're happy with shift/interleave, and don't mind the halos, then I'd just use it.

Most people watching your video aren't going to be looking at the technical quality anyway (unless the content is really boring!).

Cheers,
David.

[henryho_hk]

(unless the content is really boring!)

You remind me of those Blue-ray titles. :devil:

Suddenly I have one question on the shift correction. What does a negative "src_top" value mean for the resize? Does it mean a duplication of top pixels?

And.......... I still think that it's not a good idea to re-size the separated fields progressively as if the horizontal lines are laying on each other.