Trying to scale 1080i source to 480i

[Avisynth_challenged]

Hi everyone.

I'm working on a project to convert a 1080i MPEG2 program (a 29.97fps non-film source, so no IVTC) to 480i. The converted material is targeted for NTSC DVD.

I pulled up an old script I used several years ago, and thought I'd put it out there for scrutiny by the experts.

Okay here it is... don't laugh :)


LoadPlugin("C:\program files\avisynth 2.5\plugins\DGDecode.dll")
############


# This line loads the MPEG source
MPEG2Source("1080i_source.d2v")
############

bob()

converttoyuy2()

lanczos4resize(720, 480)

ColorMatrix(mode="Rec.709->Rec.601")

#set the desired output fieldorder
assumetff()

# reinterlace
separatefields().selectevery(4,0,3).weave()
############


I will encode to SD MPEG2 using CCE, so that's the purpose of the converttoyuy2() line.

Picture quality is paramount, so if anyone sees any problems or issues here, please please please help me to correct them. Thank you, Doom9 experts :thanks:

[Manao]

Two problems:
- NTSC is bottom field first, so no assumetff (or assumetff, but selectevery(4,1,2))
- you have to downpass the video, so instead of lanczos4resize(720,480), I would chain lanczos4resize(720,360).lanczos4resize(720,480). You'll find arguments for and against that lowpass here (http://forum.doom9.org/showthread.php?t=157767&highlight=interlac*+downpass*) and there (http://forum.doom9.org/showthread.php?t=138426&highlight=interlac*+downpass*), but I think everybody was agreeing on the fact that SD interlaced is lowpassed.

[Avisynth_challenged]

Two problems:
- NTSC is bottom field first, so no assumetff (or assumetff, but selectevery(4,1,2))
- you have to downpass the video, so instead of lanczos4resize(720,480), I would chain lanczos4resize(720,360).lanczos4resize(720,480). You'll find arguments for and against that lowpass here (http://forum.doom9.org/showthread.php?t=157767&highlight=interlac*+downpass*) and there (http://forum.doom9.org/showthread.php?t=138426&highlight=interlac*+downpass*), but I think everybody was agreeing on the fact that SD interlaced is lowpassed.

Thanks Manao. Do you think spine36resize would be better than lanczos4resize? And is bob() good enough? I thought I saw somewhere in the forum that there were better bob algorithms than just plain bob().

Thanks again for your help.

[manono]

- NTSC is bottom field first...
On DVD it can be either, but in my experience it's usually TFF. Maybe you're thinking of DV?

[Manao]

Do you think spine36resize would be better than lanczos4resizeChoose the one _you_ prefer.
I thought I saw somewhere in the forum that there were better bob algorithms than just plain bob().There are, but for the purpose of what you're doing, you don't really need them. You actually could do without bob() altogether (by resizing each field independently, then by rephasing them properly), but it's just simpler to use bob(), hardly slower, and it won't reduce the quality.
On DVD it can be either, but in my experience it's usually TFF. Maybe you're thinking of DV? You mean there are interlaced (_not_ telecined) NSTC DVD that are TFF ? I've never encountered a purely interlaced NTSC DVD, so I can't say if some are TFF, but NTSC (SD) as a standard is BFF, so it would really be strange if interlaced DVD were TFF. (There again, interlacing being such a mess, I wouldn't be too surprised either...)

[manono]

You mean there are interlaced (_not_ telecined) NSTC DVD that are TFF ?
Yes, probably something like 99% of them.
...but NTSC (SD) as a standard is BFF...
Do you have any support for that claim? This is interlaced DVD video we're talking about, right, and not DV AVI?

[henryho_hk]

As for the aspect ratio, 1920x1080 (pixel shape = 1:1) should resize to 704x480 (NTSC 16:9 - pixel shape ~= 40:33), and then be padded with 8 pixels on both sides.

Plain Bob() is okay, since (1) 1920x1080 is too slow for everything else (2) we are downsizing to such a big extent anyway.

And I've been both TFF and BFF NTSC DVDs.... so I think it does not matter.

[Motenai Yoda]

LoadPlugin("C:\program files\avisynth 2.5\plugins\DGDecode.dll")
############


# This line loads the MPEG source
MPEG2Source("1080i_source.d2v",upconv=1,icc=true, cpu=0)
############
#keeping the interlacing
separatefields().blackmanresize(720,240,taps=9).weave()

tdeint()#or another deinterlace, like needi3, what works
blackmanresize(720,480,taps=9)
colormatrix shouldn't be necessary 'cause ntsc has Rec.709 colors like HDs.

too mantain the dar:
taking into account the BT.601 coefficients, and converting square pixel-> industry square
should be 703x480 whit 8-9 pixel on the sides that can be done in rgb colorspace, or 702x480 with 8-10 px.

[TheSkiller]

As far as I know in raw analog NTSC the lines that are respresented by the bottom field in digital get scanned first, so to say NTSC is BFF.
But it is not that easy because NTSC has 486 lines and it is uncertain how the digital 480 lines get aligned in those 486 lines. If they are to be aligned in the middle with 3 dead lines on the top and bottom then the field order would change, because 3 is an uneven number, thus swapping the field order, making a TFF DVD the "optimal" source for output.

Anyway, it is not so important for DVD making because the DVD Player will handle the correct field order output according to the BFF flag on the DVD. So basically it does not matter. The DVD player can alter the field order by shifting the picture one line up or down or by phase shifting (change the pairs of fields).


And like henryho_hk and Motenai Yoda already suggested, you should resize to 704x480 for a correct aspect ratio.


colormatrix shouldn't be necessary 'cause ntsc has Rec.709 colors like HDs.Why do you think so? I disagree, if I'm not totally mistaken NTSC DVDs (and DV) use Rec.601 coefficients just like PAL and any other non-HD video source. The reason is simply because regarding the color there is no difference between PAL and NTSC in digital, so why should NTSC DVDs use different coefficients?

[manono]

colormatrix shouldn't be necessary 'cause ntsc has Rec.709 colors like HDs.
Here we go again. It most definitely is necessary. Hi-Def is Rec.709 and Standard-Def is Rec.601. Or supposed to be.

[Alex_ander]

NTSC (SD) as a standard is BFF, so it would really be strange if interlaced DVD were TFF.
It is the task of a DVD player to produce a standard analog output of what is stored. In analog form, the field order is only important for clean switching of synchronous sources (the last field of previous source and the first field of next source must be spacially different in a frame). In single source aspect, the only difference between TFF and BFF cases is that fields of a stored frame will be mapped to different halves of analog frame sync interval (these halves are recognized in sync/blank pulse structure as 'first' field and 'second' field of a frame). No spacial field shifting occurs at that, so in the end it is impossible to notice any difference between TFF/BFF digital sources.
As for field order in digital form by ITU601 (by line numbers defined there), DV camera standards safely ignore its PAL's TFF and this allows for storage of 576 complete lines (without blanling halves) as well as restoration to standard analog PAL with full 575 number of lines.

As for bob() as single-field based thing (same with pure NNEDI's). In case of a vertically sharp picture a field can't properly (without artifacts on horizontal edges) represent the full image. By sampling theorem, in the half-line case vertical spacial bandwidth must be limited to half of that of full (properly produced) frame. So a smarter bob using both fields for each new frame is desirable here.

[Alex_ander]

And like henryho_hk and Motenai Yoda already suggested, you should resize to 704x480 for a correct aspect ratio.

You can (unlikely should), but in most cases the recommended padding to 720 might be a wrong thing (unless your encoder is capable of writing to mpeg 'sequence_display_extension' - with those 704, 480 numbers). So says ITU's H.262 paper.

[Motenai Yoda]

Here we go again. It most definitely is necessary. Hi-Def is Rec.709 and Standard-Def is Rec.601. Or supposed to be.
I've told that because all ntsc r1 that I saw were Rec.709, anyway the colorimetry flag of the encoder can be set right.

[mp3dom]

SD is Rec.601 for both NTSC and PAL. If the flag is not set it could be that some software shows Rec.709 as an error. This doesn't change the fact that the color primaries should follow the Rec.601 specs. Downscaling from HD to SD requires the ColorMatrix plugin. When you capture from masters in raw uncompressed (often is UYVY 4:2:2) the NTSC resolution is 720x486 (the software then crop 6 lines, in general 4 lines from top and 2 lines from bottom or viceversa)

[Avisynth_challenged]

Wow, lotsa great information from the experts.

Here's my revised script FYI :)


# This line loads the MPEG source
MPEG2Source("1080i_source.d2v")
############

bob()

converttoyuy2()

spline36resize(704, 360).spline36resize(704, 480)

AddBorders(8,0,8,0).ColorMatrix(mode="Rec.709->Rec.601")

#set the desired output fieldorder
assumetff()

# reinterlace
separatefields().selectevery(4,0,3).weave()
############


It looks good when previewing in VirtualDub. Thank you all once again, and Happy Holidays to the Doom9 community.

[manono]

I've told that because all ntsc r1 that I saw were Rec.709, anyway the colorimetry flag of the encoder can be set right.
From what did you get that information? DGIndex? If so, update your DGIndex and read the included doc about what it says these days. If from the Colormatrix doc, it's just wrong and should be updated.

[Motenai Yoda]

Because, obviously, I was wrong, I take note.

@mp3domWhen you capture from masters in raw uncompressed (often is UYVY 4:2:2) the NTSC resolution is 720x486 (the software then crop 6 lines, in general 4 lines from top and 2 lines from bottom or viceversa)
it's the same, u're talking about capture, but in this case 6 lines are lost. and assuming that 720x486 has h.262 par then 720x480== 16/9 also when assuming to scale into 720x480 without loss from HD (then 16/9) reporting with h.262 par then the two ratios match, but with bt.601 par 720x480 == 1.82:1 so 703x480 ~ 16/9
http://img837.imageshack.us/img837/6979/h262.th.png (http://img837.imageshack.us/i/h262.png/) http://img694.imageshack.us/img694/7774/bt601.th.png (http://img694.imageshack.us/i/bt601.png/)

ps great work with inuyasha is more smooth than the other seasons!

[Mounir]

SD is Rec.601 for both NTSC and PAL.

Ntsc SD (prior to 1987) used the matrix smpte 170m, The darkest black is at 9 rgb, 255 for the brightest white.

After 1987 the smpte C (conrad) matrix was progressively used.The darkest black is at 0 rgb in such a matrix and 255 for the brightest white.

In avisynth Rec.601 = smpte 170m

For Pal:

Pal use rec709 (darkest black = 0rgb) as far as i know

Matrix Color Bars:
1) Smpte 170m (http://img405.imageshack.us/img405/329/smpte170m.jpg)
2) Smpte C (http://img32.imageshack.us/img32/3125/smpteconrad.jpg)

[Undead Sega]

Wouldnt the dumb way of doing this be taking the footage into VirtualDub and resizing it to 720x480 with its resizing filter using Lanczos with the Interlaced option checked?

[Avisynth_challenged]

Wouldnt the dumb way of doing this be taking the footage into VirtualDub and resizing it to 720x480 with its resizing filter using Lanczos with the Interlaced option checked?

This could be done, but it would add an extra colorstep conversion which would be undesirable from a PQ standpoint.

(source = YV12) > (internal conversion to RGB for VDub filters) > YUY2 (desired final avi colorspace)

The Avisynth script only converts from YV12 to YUY2.

Fewer colorspace conversions is better :)

[setarip_old]

@Avisynth_challenged

Hi!

If you'd care to perform a simple and quick non-Avisynth solution, just drop your file onto multiAVCHD (then 2 more clicks) and select "SD-DVD"...

[poisondeathray]

I would recommend using clamp=0 when colormatrix (otherwise you may clip your superbrights & darks)

ColorMatrix(mode="Rec.709->Rec.601" , clamp=0)

[Undead Sega]

This could be done, but it would add an extra colorstep conversion which would be undesirable from a PQ standpoint.

(source = YV12) > (internal conversion to RGB for VDub filters) > YUY2 (desired final avi colorspace)

The Avisynth script only converts from YV12 to YUY2.

Fewer colorspace conversions is better :)

Well, dont do any colourspace conversion then :D keep it as pure as possible, from what I'm understanding your source is YV12, you dont need to convert it to anything else as any of the Avisynth or VirtualDub filters already accepts this colourspace. A sample would be nice also :D

[manono]

I will encode to SD MPEG2 using CCE, so that's the purpose of the converttoyuy2() line.
The colorspace conversion is usually saved for the last line in the script (for CCE). It would be ConvertToYUY2(Interlaced=True). Undead Sega, he can do it in the script or allow CCE to do it for him. Like Avisynth_challenged, I prefer to do it in the script.

And do what poisondeathray said.

[Avisynth_challenged]

Thanks again everybody.

Okay, my final, final script :devil:


# This line loads the MPEG source
MPEG2Source("1080i_source.d2v")
############

bob()

spline36resize(704,360).spline36resize(704, 480)

AddBorders(8,0,8,0).ColorMatrix(mode="Rec.709->Rec.601",clamp=0)

#set the desired output fieldorder
assumetff()

# reinterlace
separatefields().selectevery(4,0,3).weave()

# YUY2 needed for CCE
converttoyuy2(Interlaced=True)
############

Still learning. Thanks a bunch for the great discussion.

[scharfis_brain]

spline36resize(704,360).spline36resize(704, 480)

why do you first scale down to 360 and afterwards scale up to 480 again?
You're just wasting resolution.

[manono]

why do you first scale down to 360 and afterwards scale up to 480 again?
He was given that in some earlier advice:

- you have to downpass the video, so instead of lanczos4resize(720,480), I would chain lanczos4resize(720,360).lanczos4resize(720,480). You'll find arguments for and against that lowpass here (http://forum.doom9.org/showthread.php?t=157767&highlight=interlac*+downpass*) and there (http://forum.doom9.org/showthread.php?t=138426&highlight=interlac*+downpass*), but I think everybody was agreeing on the fact that SD interlaced is lowpassed.

[TheSkiller]

Well, I don't think it is generally a good idea to low-pass vertically when downscaling from HD to SD (and even then I'd do it with a vertical-only resize of GaussResize(width,480, p=20) or any other p-value lower than 30).

Of course this is arguable but imo the low-pass does not do much good at all if there aren't lots of extremely detailed *static* shots in the footage. Edit: To avoid excessive line twittering which is only visible in static areas. If you're unsure, then I suggest you don't do it. The danger of throwing away resolution for no reason is too big.

[scharfis_brain]

ah, I see.
I guess it is a matter of taste, whether one likes or dislikes lowpassed video.
I dislike lowpassed video...

[pandy]

He was given that in some earlier advice:

Why to resize lobes from lanczos by antoher lanczos? better is use the kernel or Blur() in vertical...

[hanfrunz]

Without prefiltering the HD source you can get flicker in the output dvd. Just try it without the lowpass and if you see annoying flicker on a crt-tv, you may filter some scenes only or the whole clip.

regards,
hanfrunz

[pandy]

lanczos and all "sharp" resizers introduce some lobes - You can use resizer which not produce lobes or lowpass sharp resizer (but there is no sense to use sharp resizer then filter "sharpnes" out)

[henryho_hk]

spline36resize(704,360).spline36resize(704, 480)

I prefer a single bilinearresize(704, 480) call.

[pandy]

IMO Bicubic is good compromise and it is fast to - as an alternative Gaussian or Spline can be used.

[pandy]

Well, dont do any colourspace conversion then :D keep it as pure as possible

Converssion from 709 to 601 is highly advised - many devices assume for SD 601 not 709.

[*.mp4 guy]

lanczos and all "sharp" resizers introduce some lobes - You can use resizer which not produce lobes or lowpass sharp resizer (but there is no sense to use sharp resizer then filter "sharpnes" out)

This is nonsense. The problem of interlacing twitter is directly explained by sampling theory, and its well known that a perfectly "sharp" lowpass is the theoretically optimal solution. In any situation where you are using a lowpass to address a problem, you want to use the one with the sharpest cut-off that does not introduce other issues.

[Alex_ander]

... a perfectly "sharp" lowpass is the theoretically optimal solution. In any situation where you are using a lowpass to address a problem, you want to use the one with the sharpest cut-off that does not introduce other issues.

If by perfectly sharp you mean 'brickwall' linear phase lowpass filter, it is known to have sufficiently oscillating transient response with 9% first overshoot. So using it for bandwidth limitation is often worse than using a filter with as strictly limited bandwidth, but with a smoother amplitude/frequency responce near cut-off frequency. Optimal choice of a 'less sharp' filter depends on spectral content of initial picture (the less sharp is that picture, the more 'sharp' filter can be used).

[*.mp4 guy]

If by perfectly sharp you mean 'brickwall' linear phase lowpass filter...
It is the theoretically correct solution, assuming perfect conditions
...it is known to have sufficiently oscillating transient response with 9% first overshoot. So using it for bandwidth limitation is often worse than using a filter with as strictly limited bandwidth, but with a smoother amplitude/frequency responce near cut-off frequency. Optimal choice of a 'less sharp' filter depends on spectral content of initial picture (the less sharp is that picture, the more 'sharp' filter can be used).
To quote myself: "In any situation where you are using a lowpass to address a problem, you want to use the one with the sharpest cut-off that does not introduce other issues." I thought It was clear that ringing (or first lobe overshoot haloing as you reference it) is one of the predominant issues that requires use of less aggressive filters then brickwall approximations. You can't just restate what I said using more technical jargon and thereby imply that what I said was wrong. I phrased my response as I did to try to make it both easily understood and clear, not because I don't know the technical details.