Difficult-to-compress movie: The Skulls (2000)

[merlinhd]

Hi,

I've come around a movie which I'm having lots of problems to compress properly. I'm doing 2-pass compression, and the average bitrate calculated in the first one is always high, well over 3000 kbps.

I've resized the movie to 640x360, without applying any other filters - appart from cropping.

I've tried different XviD settings:

- MPEG, AQ, Trellis, QPel, no GMC, VHQ = 1.
- MPEG, AQ, Trellis, no QPel, GMC, VHQ = 4.
- H.263, AQ, Trellis, no QPel, GMC, VHQ = 4.

But it seems that the result of the first pass doesn't really depend on these settings. I thought so because of XviD deactivating some options to do a faster first pass, but I read somewhere that GMC was taken into account in the first pass too, apart from the type of quantization.

I'm encoding for a target average bitrate of 1500 kbps, but the end result is not so good. Additionally, the movie is taking much more time to compress than other movies. I'm starting to think that maybe the problem is some sort of background noise that it's possible to appreciate in the movie - I'm attaching a picture of a frame, after resize but before compression, in which it's possible to see that the movie is not really "clear".

http://www.sevillaonline.com/the_skulls_frame.jpg

Does anyone have an idea of how to deal with this? Any hints would be greatly appreciated.


Álvaro

[Sharktooth]

Use:
TemporalCleaner(5,10)
RemoveGrain(Mode=2)

Noise goes away...

[Heini011]

probably the best solution: use the MVTools 0.96.2 from Manao (http://jourdan.madism.org/~manao/?M=D)

The new MVTools 0.97 seems to be buggy: mvdenoise produces much ghosting! But MVTools 0.96.2 works great.


use the following lqf() function within your avisynth script and use the h263 matrix.


MVTools 0.96.2 Syntax:

---------------------------------
function mvdenoise2(clip) {
backv = clip.MVAnalyse(isb=true, sx=4, sy=4, pel=1, lambda=1000)
forwv = clip.MVAnalyse(isb=false, sx=4, sy=4, pel=1, lambda=1000)
return clip.MVDenoise(backv, forwv, thT=10, sadT=64, thMV=100, thSCD1=80)
}

function hqf(clip) {
# clip=clip.FluxSmoothST(2,2)
clip=clip.deen("a3d",rad=2,thrY=2,thrUV=4,min=0.5,tthY=2,tthUV=4,scd=3)
clip=clip.MSharpen(threshold=3,strength=10,mask=false,highq=true)
return clip
}
---------------------------------


(buggy) MVTools 0.97 Syntax:

---------------------------------
function mvdenoise2(clip) {
backv2 = clip.MVAnalyse(isb=true, blksize=4, pel=1, lambda=1000, delta=2)
backv1 = clip.MVAnalyse(isb=true, blksize=4, pel=1, lambda=1000, delta=1)
forwv1 = clip.MVAnalyse(isb=false, blksize=4, pel=1, lambda=1000, delta=1)
forwv2 = clip.MVAnalyse(isb=false, blksize=4, pel=1, lambda=1000, delta=2)
return clip.MVDenoise(backv1, backv2, forwv1, forwv2, thT=12, thSAD=64, thMV=100, thSCD1=80)
}

function lqf(clip) {
clip=clip.mvdenoise2()
clip=clip.deen("a3d",rad=2,thrY=2,thrUV=4,min=0.5,tthY=2,tthUV=4,scd=3)
return clip.MSharpen(threshold=5,strength=8,highq=true,mask=false)
}
---------------------------------

greetings.

[merlinhd]

Thanks for your replies! I have actually never used filters of this type before, so I'll have to look into it.

Is it possible to use them with VirtualDub? It's what I use to encode movies, I haven't used AviSynth before.

[ObiKenobi]

Originally posted by merlinhd
Thanks for your replies! I have actually never used filters of this type before, so I'll have to look into it.

Is it possible to use them with VirtualDub? It's what I use to encode movies, I haven't used AviSynth before.

Yes, avisynth is the frameserver you use to feed the video into a program like virtualdub; it doesn't do any encoding. You just open the avs file into virtualdub like you would with any other file type vdub supports.

[Blue_MiSfit]

try some other denoisers also.

I reccommend

ConvertToYUY2()
PixieDust(2)
ConvertToYV12()

(note that you have to load the plugin dustv5.dll and also loadpluginex2.dll (BEFORE dust) from the warpsharp plugin package)

plugins are available from http://www.avisynth.org/warpenterprises/

some other "basic" filters to try on their own or in combinations

undot()
unfilter(-5,-5) or -10,-10
peachsmoother()

Theres a really cool sticky in the "avisynth usage" forum that compares different denoisers. Read it and experiment with different denoisers to get a feel for avisynth filtering. This is really the key to maximizing the potential of your encode.


are you using bframes? if not try them. they give a very nice compressibility boost.

Also, try a good custom matrix, like didee's 6of9 hvs (IMHO a good choice for your encode.

[]
Also, setup a zone for your credits, and encode that zone at constant quantizer 31 in grayscale. This will save a few more bits to help out your main movie. Every little bit helps.

Also try using kopei's new 1.1 build that has support for VHQ'd bframes.

[]

[LoKi128]

Originally posted by Heini011
use the following lqf() function within your avisynth script and use the h263 matrix.


The last line in the mvdenoise2 function should have clip instead of source, I think. That whole function gives me a blocky, ghosty mess though.

[Heini011]

Hi LoKi128,

1. you right, sorry!

2. mvtools 0.97 seems to be buggy, see my first posting.

remark: the mvdenoise parameters are set for maximal denoising without (much) blocking on motion.

[merlinhd]

Hi again,

I've been playing with both AviSynth and VirtualDub filters. It's kind of interesting to see how easily it is to bring the computer to its needs as soon as you start using filters.

About this, I think I've noticed that using an AviSynth filters with VirtualDub is much slower than using native VirtualDub filters. Can anyone please confirm this?

Álvaro

[ObiKenobi]

Originally posted by merlinhd
About this, I think I've noticed that using an AviSynth filters with VirtualDub is much slower than using native VirtualDub filters. Can anyone please confirm this?

I have the opposite effect in most cases, but which avisynth filter(s) are you referring to that is slower then its vdub counterpart?

[merlinhd]

I was just using the small script that Blue_MiSfit
suggested to remove noise from a movie before compression:

ConvertToYUY2()
PixieDust(2)
ConvertToYV12()

I haven't really tried other filters to remove noise in AviSynth - the reason bein' that I don't know how to or what filters to use. But I've tried quite a few in VirtualDub and, while slow, they were not so *painfully* slow.

What filters do you recommend to remove noise? But something simple, please, so that it doesn't take ages to compress the movie but the result is good enough.

I hope you don't mind if I ask you another question. I always use VFAPI to create, from the D2V file created by DGIndex, a pseudo-AVI file that I can open in VirtualDub. At the moment I'm using the AVISource command in AviSynth to open this AVI file. Isn't there any filter that I can use to skip this step? Maybe to open directly the IFO or VOB files, or the D2V file?

Well, thanks for everything.

[PiXuS]

Originally posted by merlinhd
I was just using the small script that Blue_MiSfit
suggested to remove noise from a movie before compression:

ConvertToYUY2()
PixieDust(2)
ConvertToYV12()

I haven't really tried other filters to remove noise in AviSynth - the reason bein' that I don't know how to or what filters to use. But I've tried quite a few in VirtualDub and, while slow, they were not so *painfully* slow.

See this sticky (http://forum.doom9.org/showthread.php?s=&threadid=51181) in the AviSynth forum. You will notice PixieDust is one of the slowest. It has nothing to do with AviSynth.

What filters do you recommend to remove noise?

PixieDust ;)


I hope you don't mind if I ask you another question. I always use VFAPI to create, from the D2V file created by DGIndex, a pseudo-AVI file that I can open in VirtualDub. At the moment I'm using the AVISource command in AviSynth to open this AVI file. Isn't there any filter that I can use to skip this step? Maybe to open directly the IFO or VOB files, or the D2V file?


Use Gordian Knot and look at the generated AVS script. It will give you hints as to how you should proceed. Hint: "DGDecode.dll" and "mpeg2source".

[Mug Funky]

hehe... when you consider just how much pixiedust does, you'll realise why it's slow.

seeing as motion-compensated filtering techniques are getting so popular, i think maybe it should be implemented encoder-side somehow - after all, with motion compensating it's like you're encoding twice.

maybe something similar to inloop filtering, but somehow customizable? nah, i'm dreaming.

btw, if you're filtering with avisynth, put "fast recompress" on in virtualdub to save time and avoid an unnecessary colour conversion (which will blur things).

[Audionut]

LoadPlugin("C:\DGDecode.dll")
LoadPlugin("C:\removegrain.dll")
LoadPlugin("C:\unfilter.dll")
LoadPlugin("C:\deen.dll")
Mpeg2Source("C:\yourvideo.d2v")
removegrain()
crop(x,x,x,x)
deen()
lanczos4resize(xxx,xxx)
removegrain()
unfilter(-5,-5)


Notes. Removegrain will not process the edges of frames. Hence why it is placed before cropping.
Removegrain will only remove "small" grain.
Resizing to a smaller resolution will make grain "smaller".
Hence why place removegrain after resizing aswell.

xvid settings.
AQ on
GMC on
Custom Quant Matrix (http://forum.doom9.org/attachment.php?s=&postid=536163)
And here (http://forum.doom9.org/showthread.php?s=&threadid=85178)
And here for the results of the 900kbps quant test (http://forum.doom9.org/showthread.php?s=&threadid=84375)
Trellis on
VHQ=4/VHQ for b-frames.
Full Quality First Pass/I-frame boost=0 on second pass.
Also use zones for the credits.
In the credits zone. Quant=31.

[pelle412]

Your b-frame settings for end credits is actually going to make the end credits take more space than if you use default b-frame settings with quant 31.

[merlinhd]

Yeap, it seems to be true that using AviSynth filters with VirtualDub is faster than doing the same filtering with VirtualDub.

I cannot really tell about the real speed difference, but it seems that AviSynth is run in a different thread, while VirtualDub filters are run in the same thread as XviD. This benefits multiprocessor computers and HT processors.

[Audionut]

Originally posted by pelle412
Your b-frame settings for end credits is actually going to make the end credits take more space than if you use default b-frame settings with quant 31.

It does indeed.

Above post now edited.

[Sharktooth]

Originally posted by merlinhd
Yeap, it seems to be true that using AviSynth filters with VirtualDub is faster than doing the same filtering with VirtualDub.

I cannot really tell about the real speed difference, but it seems that AviSynth is run in a different thread, while VirtualDub filters are run in the same thread as XviD. This benefits multiprocessor computers and HT processors.
... and avisynth natively supports YV12 color space (that is far less data than RGB) so there is no color space conversion from the source to the filters as it happens with virtualdub filters (YV12-RGB-YV12).
Processing less data and avoiding unnecessary color conversions leads to more quality and better speed...