Looking for an "all-purpose" avisynth script

[bartel75]

I've been converting DVD videos for a couple of years now, and I've used lots of different combinations of resize, deinterlace, denoise, etc. filters.

Because I don't want to spend (a lot of) time finetuning the avs script for each individual conversion, I tried creating some "standard" settings.

This is what I currently use (relevant lines only):
Mpeg2Source("H:\mymovie.d2v")
TomsMoComp(-1,5,1)
GaussResize(712,528,8,0,-0,-0,p=60)
ColorMatrix(d2v="H:\mymovie.d2v")
hqdn3d(2)
TTempSmooth()

Is there any way I can improve this script? Might some filters be outdated, and/or have better alternatives? Or maybe there is something wrong with how these filters will interact?

Some (possible important) remarks:
- I always encode to x264, mostly using crf 22, or sometimes crf 20
- quality is important, but also: decent conversion speed, and relatively small file size (say, about 1200 kbps)
- TomsMoComp: when working with video that is clearly not interlaced, I skip this step, of course
- GaussResize: evidently, the values are variable, depending on the source. Please note that I don't resize; I just use this for cropping...
- ColorMatrix: not sure if I really need this, but I suppose it's fast, and it doesn't hurt

So, my question: any chance I could improve this "standardized" script?

Thanks a lot for your help!

(BTW, I haven't found any threads related to standard avs settings, but if I'm mistaken, my apologies, and thanks for pointing me in the right direction)

[Didée]

I know of exactly ONE script does fit all possible cases.

Return( last )

Anything more than that, and it won't be all-purpose.

[bartel75]

I know of exactly ONE script does fit all possible cases.

Return( last )

Anything more than that, and it won't be all-purpose.

Okay then, make that "almost-standard" then... :)

What I meant: I don't want to fiddle with the settings each time I encode a new video. Of course, if I see a video that is (for example) extremely grainy, I would adjust the settings accordingly.

But most videos (up to now) were regular PAL DVDs (movies), with the occasional exception of some old B&W footage from the 50s or 60s.

One specific example: I ALWAYS use TTempSmooth(), but maybe you guys think there's a filter that does the same, does it faster, or gives better quality without a severe time penalty?

[Guest]

Nobody here is going to support a quest for a "standardized script" ignoring the specifics of the source material.

If you are asking for suggestions for your posted script, then you will need to provide a sample of the source material it is to be used with.

[bartel75]

Nobody here is going to support a quest for a "standardized script" ignoring the specifics of the source material.

Maybe my wording was a bit off. (I kept the title of my post simple in the hope of getting some simple (read: "staightforward") replies.) Naturally, I would modify most of the script options to suit my needs. Especially with hqdn3d, I change the denoise level depending on the source and the compression rate I want to achieve.

Perhaps I could ask some questions that can be answered in a general sense.


For example: I suppose that, for just about any type of video, temporal smoothing is a good idea. Could you tell me if I might be mistaken, and there might be some specific case I overlooked, where TS is not the right thing to do?

The same goes for ColorMatrix. It's my idea that you should always use it. Am I correct?

One last example: suppose I always used PixieDust, and I talked about wanting fast conversion. Then someone could point out to me that this filter is quite cpu-intensive, and that there are faster alternatives.

I hope I'm getting my message across a little better this time...

BTW: the reason I'm not talking about the "specifics of the source material" is the following: I am trying to find a way to convert videos without having to come to the forum each time I'm doing a new encode. You guys probably have better things to do, and I want to find a way to work things out myself. But I'm getting a little overwhelmed reading about (for example) all 20 deinterlace filters mentioned on avisynth.org, let alone testing them one by one, repeating the proces for each new encode I'm starting.

Could anyone at least have their say about my two specific questions above (TS / colormatrix) ?

Thanks in advance!

[Gser]

The man comes to Doom9 for "staightforward" answers. :D You have come to the wrong place. Now first of all, all commercial movies are either shot on film which is always progressive or captured progressively with a digital camera. TomsMoComp is meant for natively interlace material - not progressive material which as been interlaced in post-process. So TFM from the TIVTV package would go a long way. And then AssumeFPS("ntsc_film") to change the fps to its original state but only when you plan on trancoding the audio into another format. Why would you use GaussResize for cropping, there is a function called Crop for that, you know. What's with all the denoising?

[BigDid]

...
Perhaps I could ask some questions that can be answered in a general sense.
...
Hi,

Problem is, avisynth is complex (and needs scripting)!

Either you do not want to script and try to use a GUI which wraps these filters in something you can manage, either you want full control and have to script.

What I mean by script is wrapping one filter after another and entering manually parameters to tune the filter/function to the desired effect; many filters, many parameters.

So as Didée as pointed out: "Anything more than that, and it won't be all-purpose".

At this point I can only point you to a very useful (IMO) thread that will give you what usual filters are used by some (estimated) members of this forum. Up to you to read/compare test these ressources. But, it will not give you a all-in-one / all-purpose solution!

It is called what are your current go-to filters:
http://forum.doom9.org/showthread.php?t=149919

Did

[Keiyakusha]

TFM from the TIVTV package would go a long way. And then AssumeFPS("ntsc_film") to change the fps to its original state but only when you plan on trancoding the audio into another format.

Emmm... what? After TIVTC, decimation should be applied, and it has nothing to do with audio.

[bartel75]

Why would you use GaussResize for cropping, there is a function called Crop for that, you know. What's with all the denoising?

GaussResize: I guess it comes from the time I used HDConvertToX. It's (AFAIK) the default crop/resize filter there, even when you don't actually resize. And when I went to manually inspecting the avs scripts this tool generated, I just left it there. Crop is of course the simpler solution in my case (as I rarely or never resize).

Denoise: good point. I should probably get my foot off the gas pedal there!

Thanks for your input...

[bartel75]

Either you do not want to script and try to use a GUI which wraps these filters in something you can manage, either you want full control and have to script.

As I just said in my previous post, I come from HDConvertToX. But I'd rather know what I'm doing, so (for example) instead of setting the de-noise filter option to "super light", I prefer to see which function is actually called, and with what parameters. Let's say I want to have 'some' control. ;)

So as Didée as pointed out: "Anything more than that, and it won't be all-purpose".
Ok ok, it won't be all-purpose, then... ;)

Thanks for the link!

[Guest]

For example: I suppose that, for just about any type of video, temporal smoothing is a good idea. Wrong. Why would you think that?

The same goes for ColorMatrix. It's my idea that you should always use it. Wrong. Why would you think that?

One last example: suppose I always used PixieDust, and I talked about wanting fast conversion. Then someone could point out to me that this filter is quite cpu-intensive, and that there are faster alternatives. There are ample threads about denoising here. But again, there are many types and degrees of noise and thus no standard prescription. And do you necessarily always want to remove film grain, for example?

I hope I'm getting my message across a little better this time... I'm afraid not.

I am trying to find a way to convert videos without having to come to the forum each time I'm doing a new encode. Then increase your knowledge base so that you can draw on it as needed rather than asking for spoonfeeding in every case.

[Blue_MiSfit]

What neuron2 says here is key:


Then increase your knowledge base so that you can draw on it as needed rather than asking for spoonfeeding in every case.


Use "no filters" as your default :) You'll learn when sources will need denoising when you're targeting a specific filesize for example, and you should spot interlacing issues when running DGIndex etc...

~MiSfit

[bartel75]

Wrong. Why would you think that? (temporal smoothing) My reasoning, based on the info found on the net: TS greatly helps compressibility, without introducing any (obvious) artifacts to the final video. To my eyes, at least... Please do correct me if there's something I forgot to take into account...

Wrong. Why would you think that? (ColorMatrix) From the filter description on avisynth.org: "ColorMatrix corrects the colors of MPEG-2 streams of dvds." As I'm working with exactly these streams all the time, it sounds like a no-brainer to me. What I couldn't find on the net though: maybe 80% of mpeg streams are already perfectly fine, and this filter has no obvious advantage?

There are ample threads about denoising here. I agree that denoising questions cannot be answered in a simple way. Luckily for me, I can do some tests, and just compare the different results myself...

Use "no filters" as your default

Blue_MiSfit, I'll follow that advice.

I'm just left with the two filters (ts & colormatrix) that seem to be 'universal'. So my remaining 'specific' question:

Why not use ts & ColorMatrix all the time? What would be a situation where it's not recommended? ColorMatrix ensures that my colors are never off, and ts ensures better compressibility, without any major disadvantages. Right? (I guess you guys will say 'wrong', but I'm curious about the specifics ;))

For the record, my main sources: this forum, and the avisynth website. Maybe there's an important source of info out there that I don't know about?

[Dogway]

hello bartel75. As you see Im newer than you, but I might have spent more time reading these pages and others as well, so I throw ya some tips :D

Colormatrix must be used only if your source colors have the wrong type. You check by mpeg2decode (info=1) I think. Note SD is 601, HD is 709. If you use hints=true then it will automatically read if a conversion is needed and fix it accordingly, but as you see, there are exceptions where there is no tag to read from, and you must eye-check yourself. But normally is right.

what is ts? temporal smoothing? Id only use a strong temporal if its temporal stable source (no shaking, not heavy action,no grain(?)), if not a low temporal should be enough.

[Gser]

Emmm... what? After TIVTC, decimation should be applied, and it has nothing to do with audio.

DECIMATION SHOULD NOT BE APPLIED TO A PAL SOURCE!! TFM is a frame matching filter - it leaves the frame count. Tdecimate removes the duplicates. In most cases anyways - I haven't encountered a single case in which it was needed. Especially since we're dealing with film sources which tend to be sped up to 25fps rather than use dummy frames.

PS. UnDot is a good filter for reducing the file size without loosing a lot of visual appeal. (Much better alternative than all that denoising anyways.)

[7ekno]

PS. UnDot is a good filter for reducing the file size without loosing a lot of visual appeal.

Doesn't RemoveGrain(mode=1) give identical results to Undot(), but execute faster due to SSE optimizations and better MT compatibility?!?

7

[Gser]

Doesn't RemoveGrain(mode=1) give identical results to Undot(), but execute faster due to SSE optimizations and better MT compatibility?!?

7

I don't know and care tbh. Undot works at least at realtime for me on HD material. And I never use it.

[txporter]

Doesn't RemoveGrain(mode=1) give identical results to Undot(), but execute faster due to SSE optimizations and better MT compatibility?!?

7

Yes, that is what I have read. I use RemoveGrain(1) rather than Undot() because it is faster.

[BigDid]

Hi,
I don't know and care tbh...
Up to you to build or not your credibility.
Doesn't RemoveGrain(mode=1) give identical results to Undot(), but execute faster due to SSE optimizations and better MT compatibility?!?

Yes it gives identical results, yes it should be faster but you usually won't notice (both realtime).

While we are on the subject :
* use of remove grain() (spatial) from weaker to stronger:
mode1 => mode2 ...
* use of degrainmedian() spatio/temp, weaker to stronger:
mode5 => mode4 => and so forth..

I prefer dgm/mode5 to removegrain for hot pixels, kind of all purpose; my opinion only:)

Did

[bartel75]

Yes, that is what I have read. I use RemoveGrain(1) rather than Undot() because it is faster.

That might be a candidate for my "all-purpose" avs script! :devil:
I always had trouble with Undot's extensive list of exotic parameters!

On a serious note: degrainmedian sounds like a good idea, too. I'll run some tests...

[Bi11]

I used to use fft3dfilter(sigma=4, sigma2=0, sigma3=0, sigma4=0 bw=32, bh=32, ow=16, oh=16, bt=5) as an "all-purpose" filter at the end of every script to increase compressibility and actually increase SSIM with x264.

However, after using fft3dfilter for several sources and encoding with x264, when I actually played the videos I saw they were softer and less sharp than the original. Other denoisers like RemoveGrain, Undot, DegrainMedian, TTempSmooth, etc..., had a similar effect but they would also give rise to various artifacts, like reducing the brightness in small eyes.
I think the word for what was removed/missing is "complexity". The same complexity AQ and Psy-RD try to retain with x264.

So now I try to avoid denoising unless it's necessary. Instead, for real-life videos, I just load the video, do the necessary filtering to make the video look flicker-free but still complex (like deblocking with deblock_QED then slight kernel shapening with LSFMod to enhance detail), then encode with x264, generally using the following parameters:
x264.exe --preset slower --tune film --aq-mode 2 --psy-rd 1:0.25
So since you're testing, try encoding the video (using the above settings with 2-pass to ensure comparable bitrates) without any post-processing and compare it with the post-processed encodes.