How2 resize 4k material to SD interlaced - avoiding aliasing and other nasty things?

[freezer]

Hi there avs gurus,

I would like to ask for some help with the following situation:

I have progressive 4k footage (from RED ONE camera (http://en.wikipedia.org/wiki/RED_Digital_Camera_Company)) and want to resize the material to PAL widescreen standard definition (interlaced).

A simple resizing filter (regardless of the algorithm tested) shows heavy aliasing with high frequency details. Bluring the picture before resizing helps a bit, but also blurs frequencies making no problems.

One solution I could think of would be kind of a low pass filter with a nice curve like roll off in the frequency domain (I guess thats what you call an antialiasing filter?).
Unfortunately I am not very script programming savy - so I need some help with this. Anyone?

If needed I can provide you an 1080p sample footage where you can see the aliasing when you do the SD conversion.

Thanks in advance for the help!

[Dark Shikari]

One idea:

1. Bob-deinterlace, with a dumb bobber.
2. Resize down to your target size.
3. Re-interlace using SeparateFields() and SelectEvery.

[freezer]

Thanks, but the material is progressive footage with a resolution of 4096x2304 px, 3x16bit, so deinterlacing doesn't make sense because the problem is not interlace flicker in the source.
I have edited the first post to reflect this better.

[Dark Shikari]

Thanks, but the material is progressive footage with a resolution of 4096x2304 px, 3x16bit, so deinterlacing doesn't make sense because the problem is not interlace flicker in the source.
I have edited the first post to reflect this better.Ah, in that case just skip step 1.

Can you give a single source image (4K) as an example for us to play with?

[Nikos]

Try:

Avisource("your.avi")
AssumeTFF() # or BFF, set the video's fieldorder correctly
Bob(0,1) # better use an adaptive bobber

Spline36resize(...)
OtherFilters...

AssumeTFF() #or BFF, set the fieldorder again because plain Bob always resets the fieldorder to BFF!!!

separatefields().selectevery(4,0,3).weave() # reinterlace is TFF if the source is TFF
# reinterlace is BFF if the source is BFF


Edit:
I think the material was interlace footage.

[MfA]

Nikos, that won't work ... the problem is that the resize filters in Avisynth are not actually resize filters, they are resamplers. For small ratios the difference isn't important, which is why this community got in the habit of simply giving both the same name, but for this kind of ratio using the avisynth "resize" filters alone will create too much aliasing.

Freezer, Gaussian blur works fine as an anti-aliasing filter ... if it's 50/60 Hz content you might want to add some extra blur vertically (because interlacing sucks, and this covers that up).

[Gavino]

the problem is that the resize filters in Avisynth are not actually resize filters, they are resamplers. For small ratios the difference isn't important, which is why this community got in the habit of simply giving both the same name, but for this kind of ratio using the avisynth "resize" filters alone will create too much aliasing.
Don't the resize filters in effect do anti-aliasing when downsizing by changing the support width of the resampling filter? Or is this too crude an approach for larger ratios?

[Sagekilla]

I always thought of resizing as resampling -- what's the difference here MfA?

[MfA]

Gavino : They have fixed support, bicubic resize can have variable amount of blur ... but it's support is not large enough for this kind of ratio.

Sagekilla : When you are merely resampling you are essentially trying to determine the point sample values of the original signal (before sampling) at different locations than the original samples. The original signal in this case though is an image prefiltered (for instance by the lens system, the CCD and the diffuser in front of that CCD) for the specific original sampling frequency. That is not the image you want to sample when resizing, you want to sample the image appropriately pre-filtered for the new sampling frequency. You can't actually get that pre-filtered image, but putting a gaussian blur over the image you do have is a good enough alternative when downsizing by large ratios (for small ratios there is enough blur in the resampling process itself).

[Gavino]

They have fixed support.
Not so - here's the code from resample_functions.cpp
double scale = double(target_width) / subrange_width;
double filter_step = min(scale, 1.0);
double filter_support = func->support() / filter_step;

So when downsizing (scale < 1), filter_support is increased accordingly.

[MfA]

Oops ... yes you are right the filter support is variable. Wonder why they don't actually allow it to grow though (because of the min function).

[Gavino]

For scale >= 1, the min ensures the filter support is unchanged. For scale < 1, the min equals scale, hence filter_support is multiplied by 1/scale, thereby increasing it.

[Blue_MiSfit]

Better do this at the end of your production workflow - since AviSynth only does 8bpc.

~MiSfit

[MfA]

Oh well, ignore everything I said then :) (Except for the vertical blur to cover up interlacing flicker.) Yes, the increased support should basically do the same as the blur already.

The OP could always try Nikos's script with a lanczos filter with lots of taps.

[IanB]

Try...
BlackManResize(704, Height(), Taps=6)
BlackManResize(Width(), 576, Taps=4)
AddBorders(8, 0, 8, 0)Adjust taps to control residual aliasing.

Or if you want more blur...
GaussResize(1696, 1152, P=20) # Adjust intermediate size and P to control blur
BlackManResize(704, 576, Taps=4)
AddBorders(8, 0, 8, 0)Maybe split horizontal and vertical (as above) to apply different levels of blur to each.

[florinandrei]

I am thinking to write a REDCODE import plugin for AviSynth. I'm not sure if I'll have time to do it, but in any case I got the NDA from RED, I haven't signed it yet, I'm still decyphering the licensing issues. An open source plugin is probably out of question, but it might be possible to release a binary-only import plugin.

If anybody else wants to look into this, by all means go ahead and do it. I'm not sure I'll actually do this project, lack of time being the main issue, but when Scarlet gets released next year, there will be plenty of people creating 3k content and looking for video processing tools, so it's probably a good idea to write such a plugin.
Just go to red.com (http://www.red.com/) or reduser.net (http://www.reduser.net/forum/) and you'll figure out how to obtain the NDA for their REDCODE SDK - it's only a matter of sending an email to RED. The SDK is free.

There's plenty of free REDCODE content (4k, 2k, maybe other resolutions too) at redrelay.net (http://redrelay.net/).

[krieger2005]

Doesn't exist there a resizer which resize in steps and add gaussianblur to remove aliasing during the resize-process? I'm not sure if this is the same i remind here but found this through google: http://forum.doom9.org/showthread.php?p=949610#post949610

Not sure if this could fit your needs, since it is in general an upsizer (??)

[2Bdecided]

Sagekilla : When you are merely resampling you are essentially trying to determine the point sample values of the original signal (before sampling) at different locations than the original samples. The original signal in this case though is an image prefiltered (for instance by the lens system, the CCD and the diffuser in front of that CCD) for the specific original sampling frequency. That is not the image you want to sample when resizing, you want to sample the image appropriately pre-filtered for the new sampling frequency. You can't actually get that pre-filtered image, but putting a gaussian blur over the image you do have is a good enough alternative when downsizing by large ratios (for small ratios there is enough blur in the resampling process itself).Sorry, but this simply isn't true.

All the AVIsynth resamplers include a pre-filter, or have a filter implicit within the operation of resizing.

I've just tested:

bicubicresize
lanczosresize
lanczos4resize
spline36resize

They are all alias-free when resampling down from 3200x3200 to 200x200. I used a torture test of horizontal, vertical, and diagonal frequency gratings. The high frequencies were reduced to uniform grey in the small version, just as they should be.

Any aliasing you see is due to having an "interlaced" output. There's a thread about that somewhere here...

EDIT: here you are:
http://forum.doom9.org/showthread.php?t=139948

That thread discusses the issue in detail. You have to pick a compromise between flickering, softness and ringing that subjectively pleases you the best. The filter from mp4guyin that thread won't be correct for 4k - maybe you can generate another filter for use at 4k (very slow), or drop your content to 1080p before filtering (better), or use one of the other methods in that thread (faster / ?better?).

Hope this helps.

Cheers,
David.

[MfA]

Sorry, but this simply isn't true.
Damn, hoped no one would belabour the point there :(

(Already said I was wrong ;))

[2Bdecided]

Damn, hoped no one would belabour the point there :(

(Already said I was wrong ;))Sorry - I missed that. I was so freaked out by the mere possibility that the resizers didn't work properly that I went straight off and checked!

At least I provided a useful link :D

Cheers,
David.

[frank]

The source 4096x2304 in comparision with dvd 720/704x576 is very oversampled - 4x and more! We don't need EVERY pixel.

First use ReduceBy2() or Tom Barry's Plugin YV12InterlacedReduceBy2().
It makes a large speed-up. Resulting frame is 2048x1152 and DEINTERLACED.
After that you can resize with a standard resizer to your wanted size.
And... do not forget colormatrix() to get the standard color space for DVD (ITU-R BT.470-2 for SD).

[freezer]

For all of you interested:
I have uploaded sample footage which shows aliasing problems when downconverted to PAL SD (look at the car grill).
The footage is 4096x2304 progressive, 8bit TGA, contains 25 frames.
Download the sample footage here (http://www.loom.at/~loomdata/private/RED4k2PAL.rar) (RAR, 225 MB)

[mikeytown2]

What framerate did you shoot at?


CoronaSequence("%6.6d.tga",gapless=true)
BlackmanResize(1024, Height(), Taps=6)
BlackmanResize(Width(), 576, Taps=4)


Compare Outputs
http://img145.imageshack.us/img145/5735/frame0blackmanresizejp8.th.jpg (http://img145.imageshack.us/my.php?image=frame0blackmanresizejp8.jpg) - BlackmanResize

http://img222.imageshack.us/img222/1379/frame0spline64resizeaf1.th.jpg (http://img222.imageshack.us/my.php?image=frame0spline64resizeaf1.jpg) - Spline64Resize

http://img295.imageshack.us/img295/6861/frame0bilinearresizevz2.th.jpg (http://img295.imageshack.us/my.php?image=frame0bilinearresizevz2.jpg) - BilinearResize

http://img257.imageshack.us/img257/2632/frame0pointresizedy9.th.jpg (http://img257.imageshack.us/my.php?image=frame0pointresizedy9.jpg) - PointResize

http://img296.imageshack.us/img296/7757/frame0orginalxt1.th.jpg (http://img296.imageshack.us/my.php?image=frame0orginalxt1.jpg) - Original Frame


CoronaSequence() (http://forum.doom9.org/showthread.php?t=109997)

[IanB]

@freezer,

Yes the resizers are tending to produce a moire pattern in the grill. They all seem a little keen to sharpen and are compromising the high frequency roll off of the filter.

A sharpish GaussResize seems to give the greatest control over the effect (it and bilinear are our only resizers with no negative convolution coefficients and so have no sharpening).

To my eye, a P around 50 looks reasonable, but you should access it yourself....
A=Spline36Resize(704, 576)
B=GaussResize(704, 576, P=35)
C=GaussResize(704, 576, P=50)
D=GaussResize(704, 576, P=65)
E=BilinearResize(704, 576)
F=PointResize(704, 576)

Interleave(A, B, C, D, E, F)

# Simulate DVD player pixel aspect correction
BicubicResize(1024, 576) # BilinearResize(1024, 576)I have included the Bicubic output upscale to correct the displayed aspect of the image to emulate what a real DVD player might do.

Also Gamma is possibly playing a part here. The calculations in the resizer really should be done on linear values, but for performance reasons they use uncorrected addition.

Unfortunately we only have 8 bits to play with so the code below suffers from bit squeezing. I used my usual trick with a PC matrix and no coring to maximise the precision but it is still degraded. I can see improved moire pattern reduction, but the image overall is less good.... # RGB source
Levels(0, 1/2.2, 255, 0, 255) # Make linear
ConvertToYV12(matrix="PC.709")
GaussResize(704, 576,p=50)
Levels(0, 2.2, 255, 16, 235, False) # Reapply gamma correction and reset to Rec.709 levels


If you need the absolute Rolls Royce result then resize the original images with another tool with a high bit depth gamma corrected resizer.

[mikeytown2]

Another tool in order to achieve the "Rolls Royce" result ImageMagick (http://www.imagemagick.org/).

http://www.imagemagick.org/Usage/resize/

[2Bdecided]

Compare Outputs
http://img145.imageshack.us/img145/5735/frame0blackmanresizejp8.th.jpg (http://img145.imageshack.us/my.php?image=frame0blackmanresizejp8.jpg) - BlackmanResizeI think you'd have to be a very picky individual to dislike this.

I normally use spline36, lanczos, or bicubic - typically followed by limited sharpen faster.

What does imagemagik do to this image?

[frank]

1024 is no PAL size.
Moire pattern in the grill comes with horizontal shrinking to 704/720.
Good compromise of filtering is Spline36Resize or the old standard BicubicResize.
(b=c=0.33)

# 4096x2304
CoronaSequence("d:\red4k2pal\%6.6d.tga",sort=1,stop=25)
# 2048x1152
ReduceBy2()
# 1024x576
ReduceBy2()

BicubicResize(704,576)

[Comatose]

Sure it is. When you're at 1024x576, you then downsize to 720x576 (anamorphic) for the DVD.
1024x576 is the working resolution since you need to have the correct AR while editing the movie. (you know, so you can overlay stuff and they'll be the correct AR when the DVD later goes 720x576 -> 1024x576)

What framerate did you shoot at?
Well, there are 25 frames and this is going to PAL, so 25 fps?

[mikeytown2]

Well, there are 25 frames and this is going to PAL, so 25 fps?

25i needs 50p if you want smooth motion.

[Comatose]

Just coding it as interlaced (while leaving it progressive) should be good enough. I know of TV stations who do this, even with NTSC stuff.

But if he doesn't wanna do that, wouldn't just ChangeFPS(50) then 3:2 pulldown be good enough? >_>

I mean, R1 movies are certainly not shot at 59.94...

[Blue_MiSfit]

It's probably 24p (guessing).

~MiSfit

[freezer]

Yes, the original shot was 24fps, because the RED One is a digital cinema camera.
The material will be 25p coded as interlaced 50i - as usual with that type of source.

@ 2Bdecided
You have to watch the moving output on a PAL monitor to see the problems in its full glory. IanB knows what I am talking about.

@ Comatose
That is not correct - every professional software supports displays PAR and has no problem in dealing with that and different PARs at the same time. So you do not scale twice (1024x576 and 720x576) and depend on the quality of the authoring software's scaler. You just go straight to 720x576 anamorph.

@ all
Thanks for your contribution! I am going to do some extended tests on this and let you know the results. So far I like to hear as many input as you can think of.
There exists expensive hardware for downscaling which is perfectly capable of a sharp result and yet it lacks the aliasing problems. It would be great to mimick that in software - time is no matter, just quality.

[mikeytown2]

@freezer
How many frames do you have to process? ImageMagick has no gui but I've made a simple front end for it in the past with autoit (http://www.autoitscript.com/autoit3/). Having a gui is nice when your processing a lot of files. If I get some time, I'll run the short clip and post some results in a little bit. Because of all the options, ImageMagick could output the best quality.

EDIT:
AA before resize? my guess is that's what the hardware ones are doing.

[2Bdecided]

@ 2Bdecided
You have to watch the moving output on a PAL monitor to see the problems in its full glory.If you need movement, rather than a still frame, to reveal this "aliasing" then that's a different thing entirely - and pretty much unavoidable.

@ all
There exists expensive hardware for downscaling which is perfectly capable of a sharp result and yet it lacks the aliasing problems. It would be great to mimick that in software - time is no matter, just quality.I wonder if you can send test signals through such hardware to tease out the filter coefficients? Difficult, since the filtering is mainly before the resize, and the process may be non-linear.

Cheers,
David.

[henryho_hk]

You have a 4096x2304x24fps progressive video. And you want to convert it to 704x576x25i. Don't we need 48p/50p source for this?

A quick and dirty method popped out from my mind is:

Global GlbNewWidth = 704
Global GlbNewHeight = 576

function icResize(clip "source", int "newwidth", int "newheight", bool "discardhalf")
{
discardhalf = default(discardhalf, false)

src1=source.separatefields()
src2=discardhalf ? src1.selectevery(4, 0, 3) : src1
Shift = (src2.GetParity() ? -0.25 : 0.25) * (src2.Height()/Float(newheight/2) - 1.0)

E = src2.SelectEven() .Spline36Resize(NewWidth, NewHeight/2, 0, Shift)
O = src2.SelectOdd() .Spline36Resize(NewWidth, NewHeight/2, 0, -Shift)
Echr = src2.SelectEven() .Spline36Resize(NewWidth, NewHeight/2, 0, 2*Shift)
Ochr = src2.SelectOdd() .Spline36Resize(NewWidth, NewHeight/2, 0, -2*shift)

dst1=Interleave(E, O)
dst2=dst1.IsYV12() ? MergeChroma(Interleave(Echr, Ochr)) : dst1
dst2.Weave()
}

src=coronasequence("%6.6d.tga",sort=1,stop=25).assumefps(25).converttoyuy2().assumebff()
# bilinearresize
#rz1=src.bilinearresize(GlbNewWidth, GlbNewHeight) #.converttoyv12(interlaced=true)
rz2=src.icResize(GlbNewWidth, GlbNewHeight) #.converttoyv12(interlaced=true)
#subtract(rz1, rz2).Levels(10, 1, 180, 0, 255)

return rz2


Change ".assumefps(25).convertfps(50)" to something more sensible. But the whole idea of resizing is there.

[mikeytown2]

Been playing around with ImageMagick.

http://img295.imageshack.us/img295/3/29193735qw9.th.png (http://img295.imageshack.us/my.php?image=29193735qw9.png) - Mitchell 8 Lobes
http://img65.imageshack.us/img65/7780/43074710hd1.th.png (http://img65.imageshack.us/my.php?image=43074710hd1.png) - Hanning 16 Lobes
http://img505.imageshack.us/img505/9316/76663604mg3.th.png (http://img505.imageshack.us/my.php?image=76663604mg3.png) - Hanning 4 Lobes
http://img300.imageshack.us/img300/5430/43258594uy4.th.png (http://img300.imageshack.us/my.php?image=43258594uy4.png) - Cubic 4 Lobes
http://img140.imageshack.us/img140/6148/24433341nj0.th.png (http://img140.imageshack.us/my.php?image=24433341nj0.png) - Lagrange 8*2 Lobes
http://img393.imageshack.us/img393/5148/85144072rw1.th.png (http://img393.imageshack.us/my.php?image=85144072rw1.png) - Lagrange 8*2 Lobes, Support 6

Cubic and Lagrange seem to work, thus go for Lagrange