Avisynth has better Scalers built in?

[demistate]

I was at NAB 2010 this year, a show for professional broadcasters and asked Adobe, Telestream, and Mainconcept about the pre-processing abilities in their encoding software.

Grain removal was almost non existent in their products. They recommended a simple gaussian blur to help smooth it out.

All three of the pay applications I described above use Blinear as their "highest quality" scale. There were no other scaling methods. I personally find Spline36 and Lancos4 blows bilinear out of the water.

Adobe Media Encoder, Telestream Episode Pro, and Mainconcept encoders are $1000+ encoding softwares, and Avisynth does a better job pre-processing than these "all in one" software packages do.

Pair it with a free encoder like x264 and you're outperforming the most trusted software packages in the industry. In my experience, pre-processing is 80% of the encode quality.

I'm still going to stick to Avisynth for pre-processing even if I do use pay encoders to make the final crunch.

Most broadcasters don't even know about Avisynth or x264. Most of the posters here would be able to net really good jobs at professional post houses that compress films/tv shows for places like iTunes and other digital distribution networks just from their avisynth skills.

I'm just amazed that the free tools are better than the expensive, supported, pay tools.

[Lyris]

Grain removal was almost non existent in their products. They recommended a simple gaussian blur to help smooth it out.

Seriously?

:(

I'm not surprised that a lot of people haven't heard of AVISynth. From what I've seen and heard, the industry seems to operate on a "time is money" basis and doesn't really have time to write scripts or experiment with different filter chains. They like all-in-one hardware solutions, though.

However, there is an area where I think that may be changing. Since there are no hardware AVC encoders for Blu-ray production, a host of people who previously used hardware encoders to produce DVDs (directly from VTR playback I believe) are getting into the habit of capturing files from tape then encoding via software. I explained what AVISynth could do with that captured file to some other DVD/BD authors at a Sonic Solutions training event I attended in London last year, and got a few nodding heads. Sonic's people certainly knew of it :)

[Dark Shikari]

I was at NAB 2010 this year, a show for professional broadcasters and asked Adobe, Telestream, and Mainconcept about the pre-processing abilities in their encoding software.

Grain removal was almost non existent in their products. They recommended a simple gaussian blur to help smooth it out.

All three of the pay applications I described above use Blinear as their "highest quality" scale. There were no other scaling methods. I personally find Spline36 and Lancos4 blows bilinear out of the water.I don't believe this. Maybe bicubic, but bilinear is practically the worst you can possibly do for scalers besides point scaling. Even Photoshop offers bicubic.

[Blue_MiSfit]

Yes. All the encoding packages I'm personally familiar with can do bicubic or bilinear. More exotic options like lanczos and spline etc are not frequently encountered.

If you go to Snell or Terranex, you will find powerful algorithms backed by FPGAs for high quality upconversion. This is real time, but very expensive.

Avisynth is awesome, and free, but not real time ;)

~MiSfit

[lych_necross]

I don't believe this. Maybe bicubic, but bilinear is practically the worst you can possibly do for scalers besides point scaling. Even Photoshop offers bicubic.
Now thats not true about bilinear. It is a perfectly fine scaler (especially for down scaling). In fact, depending on the source material, bilinear scaling might be better because of the smoother images it tends to produce where as a bicubic scaler might over sharpen the image. It just depends on a person's preferences.

[Dark Shikari]

Now thats not true about bilinear. It is a perfectly fine scaler (especially for down scaling). In fact, depending on the source material, bilinear scaling might be better because of the smoother images it tends to produce where as a bicubic scaler might over sharpen the image.The sharpness of a bicubic scaler is adjustable.

[demistate]

I don't believe this. Maybe bicubic, but bilinear is practically the worst you can possibly do for scalers besides point scaling. Even Photoshop offers bicubic.

A huge misconception about Adobe apps is that they share some sort of common code. In reality the After Effects guys don't know what the Premiere Guys are doing. I had to ask separate engineers what kind of scaling was going on for each product. The After Effects engineer was reluctant to tell me probably because he knows that bilinear isn't the greatest in the world.

So while Photoshop does have bicubic scaling, After Effects and Premiere Pro DON'T. I don't know of many people using Photoshop to pre-process videos (even though it is possible).

I almost threw up when the Telestream pro told me that I should scale a video twice with their high quality bilinear to get a "super high-quality" HD to SD downscale. (Scale by half once, and scale that output by half again)

I wonder if these professional compression software companies are worried about licensing other scaling methods someone could have a claim to it. Crazy US patent laws.

[wonkey_monkey]

I almost threw up when the Telestream pro told me that I should scale a video twice with their high quality bilinear to get a "super high-quality" HD to SD downscale. (Scale by half once, and scale that output by half again)

That sounds pretty reasonable for a standard bilinear filter :confused:

David

[Keiyakusha]

So while Photoshop does have bicubic scaling, After Effects and Premiere Pro DON'T. I don't know of many people using Photoshop to pre-process videos (even though it is possible).

Speaking about scalers... the one in AE is so crappy so its sometimes impossible to work with it. Especially when you working with sharp CG footage, when camera flies through some raster object which is equal to 4x and more upscaling... It looks very bad.

[NerdWithNoLife]

Most broadcasters don't even know about Avisynth or x264. Most of the posters here would be able to net really good jobs at professional post houses that compress films/tv shows for places like iTunes and other digital distribution networks just from their avisynth skills.

I'm just amazed that the free tools are better than the expensive, supported, pay tools.
Absolutely! I'm one of the dummies here and I just went from being a postman to being chief engineer for a video production company. AviSynth has been more valuable than my college education in this field - seriously. (I just TGMC'd a Justin Bieber concert. That's probably a first.)

I also agree that in the business there's not always time, so you have to pay to let an expensive program think for you. But it's still good to have an idea of how to configure the stuff.

There was once a thread discussing the possibility of integrating AviSynth with hardware, and I would LOVE to see that. It is possible to run scripts live through ffdshow, and with live input. How neat it would be to have LiveVideoSource, or something like that to pull live video from a camera. I'd use it.

[demistate]

Absolutely! I'm one of the dummies here and I just went from being a postman to being chief engineer for a video production company. AviSynth has been more valuable than my college education in this field - seriously. (I just TGMC'd a Justin Bieber concert. That's probably a first.)

I also agree that in the business there's not always time, so you have to pay to let an expensive program think for you. But it's still good to have an idea of how to configure the stuff.

There was once a thread discussing the possibility of integrating AviSynth with hardware, and I would LOVE to see that. It is possible to run scripts live through ffdshow, and with live input. How neat it would be to have LiveVideoSource, or something like that to pull live video from a camera. I'd use it.


Avisynth hardware? That would be amazing.

I will admit that Avisynth's weakness is sometimes it's lack of speed. It's not very efficent when you parse AviSynth scrips through Premiere Pro, but the results do look nice :)

[SynchronousArts]

Avisynth hardware? That would be amazing.

I will admit that Avisynth's weakness is sometimes it's lack of speed. It's not very efficent when you parse AviSynth scrips through Premiere Pro, but the results do look nice :)

I've found AVISynth to be faster on a per process comparison, especially with MT based scripts.

The 8 bit limit is what stops many from using it in professional environments. Most of my use is for downconversions for web videos so I don't worry much.

SA

[Blue_MiSfit]

Lack of 10 bit only really hurts while in the post chain. After the media is finished up, you're good to go.

I take 10 bit ProRes files ALL DAY LONG into AviSynth, and it just chops off the extra data :p

You'd think it would look horrible, but most of the time it looks fine. Would I be a happy nerd with nice 10->8 dithering? Absolutely!!

Still, an AviSynth (and x264 / HC) powered encoder a-la Rhozet Carbon Coder / Telestream Episode Pro / Flip Factory would be absolutely revolutionary to the "professionals" that use said products.

Show them a 480i -> 1080i upconvert from their Rhozet farm, and then show them a 480i -> 1080i upconvert using TGMC (plus maybe NNEDI2 for the scaling). Bricks will be shat. Then show them a 4mbps 1080p H.264 file generated by MainConcept, and the same video generated by x264. Larger bricks will be shat.

I've done it :)

~MiSfit

[NerdWithNoLife]

8bpc may be a hit for compositing in RGB, but for live hardware deinterlacing (i.e. live web streaming from interlaced input), I must quote Blue_MiSfit: "Bricks will be shat."

[roozhou]

I don't believe this. Maybe bicubic, but bilinear is practically the worst you can possibly do for scalers besides point scaling. Even Photoshop offers bicubic.
IIRC MainConcept's SDK also provides a bicubic resizer that smartly picks number of taps based on source and target resolution. It also provides a way of doing resizing and color space conversion in the same time, similar to what swscale does.

[Dogway]

Except a bit of aliasing I find bilinear near perfect, it preserves detail, and doesnt create halo. Better indeed if used in float point (http://forum.doom9.org/showthread.php?t=158835).
Please compare spline64 vs PS float bilinear downsize by 2.

Spline64
http://i212.photobucket.com/albums/cc35/Dogway/Misc/spline64.png

Bilinear
http://i212.photobucket.com/albums/cc35/Dogway/Misc/bilinearps.png


The automttap3 (http://forum.doom9.org/showthread.php?p=1211873#post1211873) function by *.mp4 guy deals with details quite well through adaptive multitap but in the other hand has severe halo issues on high contrasts areas.

Another question is about why bilinear in Avisynth is so different than Photoshop's.
I made a layer comparison in photoshop:

13Mb Mediafire
http://www.mediafire.com/?3jlyiz94au3v2w8

[poisondeathray]

Another question is about why bilinear in Avisynth is so different than Photoshop's.
I made a layer comparison in photoshop:

13Mb Mediafire
http://www.mediafire.com/?3jlyiz94au3v2w8

If I'm understanding what you did correctly, your avisynth bilinear resize was done in YV12, not RGB (less color information). There might be other reasons

EDIT: Yes, I can confirm what you are seeing. It's almost as if avisynth's bilinear is shifting pixels to the left, making it look softer

There seem to be differences in the other algorithms as well... PS's bicubic looks similar to avisynths' spline36

[Dogway]

If I'm understanding what you did correctly, your avisynth bilinear resize was done in YV12, not RGB (less color information)

true, I didnt realise, but only that doesnt make such differencies :/
and yes bicubic was really similar to spline36, I thought this kind of algorithms were somewhat universal.
Now having a second look, looks like avisynth's bilinear is just as photoshop's but happy antialiased.

[Gavino]

It's almost as if avisynth's bilinear is shifting pixels to the left, making it look softer
Avisynth's resizers preserve the centre of the image.
I don't know what Photoshop does, but if it instead preserves the top left corner, Avisynth's image will be relatively shifted left (on downsizing).

Also, as Dogway says, Avisynth downsizing includes antialiasing (by extending filter width).

[poisondeathray]

ok thanks that would explain it

is there anyway to adjust the AA during downsizing in avisynth ? I couldn't find it in the docs

[Gavino]

is there anyway to adjust the AA during downsizing in avisynth ? I couldn't find it in the docs
It's fixed and depends on ratio of input and output resolutions.
For details see http://avisynth.org/mediawiki/Resampling.

[Dogway]

I went for an empiric test using pattern plates:
downsize to half only 1 time

source
http://i212.photobucket.com/albums/cc35/Dogway/Misc/zuv42a09.png
spline648AVSRGB.png bilinear8AVSRGB.png bilinear8PSRGB.png bilinearfloatPSRGB.png
http://i212.photobucket.com/albums/cc35/Dogway/Misc/spline64AVSRGB.png http://i212.photobucket.com/albums/cc35/Dogway/Misc/bilinear8AVSRGB.png http://i212.photobucket.com/albums/cc35/Dogway/Misc/bilinear8PSRGB.png http://i212.photobucket.com/albums/cc35/Dogway/Misc/bilinearfloatPSRGB.png

[*.mp4 guy]

I'm afraid you are misinterpreting the results of that test. Sampling theory makes it explicitly clear that after a 2:1 reduction in both dimensions as is the case here, only the lowest 4th of the frequency spectrum of the original image can be maintained, everything else is aliasing. In this case what that means is that having having anything outside of a central square region 1/4th the size of the total image in the downscaled output is undesirable, and indicative of aliasing or ringing artifacts. When your results are understood in this light, spline64 is empirically best, and bilinear worst.

A nearly ideal result (http://www.imagebam.com/image/4f18d1118039694)
Lanczos60 (note the extensive ringing, and the edge discontinuity) (http://www.imagebam.com/image/d3f9f7118039714)
automttap3 (it trades lanczos60's ringing for aliasing, but is otherwise similar) (http://www.imagebam.com/image/d44692118039733)

[Dogway]

*.mp4 guy my respect.

My empiric test wasnt a mathematic one, rather a psychovisual test which enforces my previous thoughts. What I see here is that at one step resize, bilinear offers me a more expectable result than anything else, and in my other post you could also see that bilinear is the only one that doesn't create haloes. It also shows that it preserves (or emules) more detail on low frequency detail than other scaling methods. That is, the central circles could be slightly worse but it keeps consistency/detail/phantasy further away the 4th concentric. And here I am recognising its aliasing issues, which is proven out in numerous tests along the webs through multiple steps resizing, but all in all I think of it like a "slight jaggy edges vs halo"


EDIT: I just did an ssim check on these patterns and bilinear gave slightly better results, maybe coincidental(?) but surprising.


I want to share a script I use for comparing resizers in AVS:
SOURCE

x=width*2
y=height*2

A=PointResize(x, y)
B=BilinearResize(x, y)
C=Spline16Resize(x, y)
C1=Spline36Resize(x, y)
C2=Spline64Resize(x, y)
Z=sincresize(factor=2,window=2) # upsize only, comment subtitle and this line out if downscale
D=blackmanresize(x, y)
E=LanczosResize(x, y)
E1=Lanczos4Resize(x, y)
E2=LanczosResize(x, y, Taps=8)
G=GaussResize(x, y, p=0.1)
G1=GaussResize(x, y)
G2=GaussResize(x, y, p=100)
H=BicubicResize(x, y)
H1=BicubicResize(x, y, 0.3782, 0.3109)
H2=BicubicResize(x, y, 0, 0.5)
H3=BicubicResize(x, y, 0, 0)
H4=BicubicResize(x, y, 1, 0)
H5=BicubicResize(x, y, 0, 1)
I=nnedi3_rpow2(rfactor=2,cshift="spline64resize",qual=2,fwidth=x,fheight=y) # upsize only, but no need to comment out if downscale
I1=eedi3_rpow2(rfactor=2,cshift="spline64resize",hp=false,fwidth=x,fheight=y) # upsize only, but no need to comment out if downscale
W=loss() # upsize only, comment subtitle and this line out if downscale
K=automttap3(dx=x, dy=y)
# L=Lagrange()
# M=wdresize()
#others?


Interleave\
(A.subtitle("NearestNeighbor")\
,B.subtitle("Bilinear")\
,C.subtitle("Spline16")\
,C1.subtitle("Spline36")\
,C2.subtitle("Spline64")\
,Z.subtitle("sinc")\
,D.subtitle("Blackman")\
,E.subtitle("Lanczos")\
,E1.subtitle("Lanczos4")\
,E2.subtitle("Lanczos8")\
,G.subtitle("Gauss")\
,G1.subtitle("Gauss2")\
,G2.subtitle("Gauss3")\
,H.subtitle("Mitchell-Netravali")\
,H1.subtitle("Robidoux")\
,H2.subtitle("Catmull-Rom")\
,H3.subtitle("Hermite")\
,H4.subtitle("B-Spline")\
,H5.subtitle("Bicubic")\
,I.subtitle("nnedi3")\
,I1.subtitle("eedi3")\
,W.subtitle("lossless")\
,K.subtitle("automttap3-lanczos"))
#,L=subtitle("Lagrange")
#,M.subtitle("wdresize")
#others?

function loss(last){
input=last
up2 = input.lanczosresize(width*2,height*2, 0.25,0.25) # for RGB: 0.25,-0.25 (RGB is stored upside down)
down = up2.pointresize(width,height)
up2 = up2.MergeChroma(input.lanczosresize(width*2,height*2, 0.5,0.5))
return up2.addborders(1,1,0,0)
}

As for mathematical can I ask you how did you achieve the ideal result resize? matlab, avisynth?

[*.mp4 guy]

Interpolation is one of the few areas I would trust PSNR more then SSIM. All interpolators I am aware of are PSNR/MSE optimized, and overall I think this is probably the correct approach, though different images may look better with different interpolators, in aggregate the highest PSNR interpolators win out.

[*.mp4 guy]

This will probably be the first time I've double posted, in this case I'm doing it because Dogway edited his post with a question after I had already posted a reply to said edited post, and I want to make sure that its clear that I'm not blowing off the question.

As for mathematical can I ask you how did you achieve the ideal result resize? matlab, avisynth?

First off, the result is not ideal, its just the best I could manage.Also, after I posted the examples, I played around with lanczos a bit more, and it does better in this case with about ~7 taps, still not as good as the example I posted, but its a lot closer then it was at 60.

To actually answer the question, it was done in avisynth, and it actually is more impressive on real world content, because it doesn't ring on transients, (which don't exist in the posted example). Usually I would just post the code, but its such a kludged mess compared to what I want it to be, and it really deserves a proper implementation.

[Dogway]

It only was a nominal indication of the subject, not pointing it was ideal. As I said I shift between PS float bilinear and AVS spline64 when resizing images, I recently discovered the multitap function you made and liked it a lot. Not only the image texture is similar to bilinear's (unlike bicubic) but as you state is also "correct". The only thing that refrains me from blindly use is the need of yv12 content input (can't help I guess), and some repetition halo, in certain circumstances.

Normally I don't see avisynth as a fast tool for image processing, but rather biased to quality, that's where all my arguments go aside any specific practical reason. But if you ask me what Id prefer out of "ringing vs some aliasing" Id say aliasing all the way.

[markanini]

Hey *.mp4 guy, what kind of resizing ratios is automttap3 optimized for?

[SilaSurfer]

I just started reencoding in Bluray world going from 1080p source to 720p x264. Which resizer would you guys recommend? With respect to Mp4guy, AutoMTtap3 is just too slow.

[*.mp4 guy]

For horizontal scaling of analog captures, and any upscaling of blurry source material, high tap (7 works well and is pretty fast) lanczos. If you don't know what to use, spline36/lanczos3. If you need less ringing, lanczos2/neutral bicubic. Absolutely no ringing, gauss, bilinear (avisynth's bilinear has larger support then normal, and is therefore not equivelent to box for downscaling), box, point (in order of ability to remove aliasing, gauss being capable of removing all aliassing, but being blurry when it does and slower then the others, point removing no aliasing at all). Don't use point or box unless you know what you are doing, they can cause unacceptable levels of aliasing, and point is basically guaranteed to completely munge your source, but it is fast.

There are a lot of other good options out there, many of them superior to those listed here in certain situations, but anything capable of significantly better performance then the listed interpolators is also significantly slower.

[edit]
Hey *.mp4 guy, what kind of resizing ratios is automttap3 optimized for?
It works for anything you would use lanczos for. It can perform marginally better if you use integer resizing ratios though, eg 2:1 downscaling. If you want to really bring out the detail, and you have a really high res source, downscaling in several 2:1 steps, using limitedsharpenfaster(smode=4, ss_x=1, ss_y=1, strength=200) before each downscale can look pretty impressive.

[SilaSurfer]

Thanks Mp4Guy for nice explenation about avisynth resamplers. I've been searching around the forum for downscaling HD material. What do you think about using for example Blackman instead of Lanczos but using diffrent tap values for both width and height like this

BlackmanResize(720, height, taps=6)
BlackmanResize(width, 576, taps=4)

This was proposed by IanB in a thread I found. One should change the number of taps to control aliasing.

BTW LimitedSharpenFaster trick is awasome, using just Spline36Resize with it doing two downscales really brought out detail in final output.

1920, 800
Limitedsharpenfaster(smode=4, ss_x=1, ss_y=1, strength=200)
Spline36Resize(1440,608)
Limitedsharpenfaster(smode=4, ss_x=1, ss_y=1, strength=200)
Spline36Resize(720,304)

I'm going to test this with Gauss and Bilinear since those are soft resizers. One could also tweak the strength value of LSF. Very nice Mp4Guy!

[markanini]

Thanks for the trick! I'm working with megapixel still images that end up as web-size. I ended up setting LSF strength to 100 without really loosing anything yet avoiding some minor artefacting. Looks really nice!
Just for fun I tried spline144 which didn't seem add or take away much from this multi-step method. I usually find spline144 best for one step resizing. Lanczos also comes out sometimes, it has a way of giving a flattering look but susceptible to ugly ringing artefacts on some sources. Nice to have an other fine tool in ones arsenal :)

@Silasurfer: Blackman is like a sibling of Lanczos, where Lanczos rings blackman aliases instead.

[SilaSurfer]

Yeah LSF strength went down to 100 for me too. I think that maybe minor prebluring before LSF would work well in avoiding artefacts. I never heard of Spline144 before.

[markanini]

It will come up in a forum search. Multi step resizing does a lot compared to single step even without LSF and spline144 adds a noticeable increase in fine detail. FWIW

[SilaSurfer]

Thanks markanini I will look it up.

[Dogway]

Spline144 sure brings up detail, but has an overshoot feel to it. Like not fine but thickened detail.

[markanini]

I suppose you're talking about an effect simmilar to sincresize taps=2?

[Dogway]

It could be since sincresize is prone to ringing. But I use v.2.57 no sincresize there.

[markanini]

Well, spline144 rings nothing like Lanczos4 but I admitt it's not an entirely natural effect and it may very well be thickening fine detail.
The benefit I see is something simmilar to this: http://www.flickr.com/photos/amica999/5201766883/

[markanini]

I just tried downscaling a 4000x3000 original still image to 500x375 with automttap3 *0.5 at a time and it looks gorgeous. Really makes the thickening of detail in Spline144 Dogway talked about obvious. If one needs fast and high quality downscaling multi-step spline36/64 is'nt bad at all, percieved resolution is not far from lanczos4 and automttap3 and artefacts are avoided.

[SilaSurfer]

@markanini

How many steps did you use with Spline32/64?

[markanini]

I just downscaled x0.5 at a time until I reached the desired resolution

[SilaSurfer]

I don't think it can reach AutoMttap3 from Mp4Guy. AutoMttap3 as far as I understant it is a resizer which prevents ringing and aliasing while preserving that sharp image. Also I found that LSF trick that came from Mp4Guy is for use with AutoMttap3 and not standard resizers in Avisynth since those will ring like hell with all of that sharpening applied before. So if you want really sharp HD->SD workflow and speed is not an issue go for it otherwise with standard resizers more speed but soft image.