HDTV->XviD encoding perfomance a.k.a. "try to do it realtime"

[OUTPinged_]

Yay, finally got 2 HDTV clips and here are some benchmarks, i think some people would be interested.

2 sample clips were used, both are 1080i material. "Plane" clip was a cut of "emotion" clip, "apple" - part of "crawford" clip.

PC is p3-840@BX with PC133 ram.

I used hdtv2mpeg2 and dvd2avi to feed a stream to avisynth. Nic's dvd2avi didnt work for me. :-(

direct mpeg2 decoding Plane: 8.33fps, Apple: 6.87fps.

That is exactly 6 times slower than same operation done using DVD source, so mpeg2dec3 is a great decoder and handles hdtv resolution flawlessly.

Now I added Decomb 4.05b2 and benchmarked again. Telecide(guide=1,gthresh=100,post=false) and Decimate(cycle=5) were used.

ivtc Plane: 4.29fps, Apple: 3.72fps

It seemed that ivtc was flawless and reconstructed everything perfectly. I expected the speed drop to be much higher, but Decomb shows us some excellent speed and slows down the process only 2 times vs 4 times slowdown i experienced working with dvd source. It may be that Decomb's perfomance is capped by memory throughput already, if somebody can confirm that please do it.

1080p is pretty pointless as it wouldnt play in divx on current pcs anyway. So here come benchmarks of resizers:


Plane Apple
lanczos 2.98 2.57
bilinear 3.46 2.88
bicubic 3.26 2.69


16% difference in speed between bilinear and lanczos.

Still too fast? Let's add postprocessing. (YAY!)

The source for "plane" is pretty clean, the only complains are that it is full of blocks.

As I expected, H+V luma deblocking enabled didnt things look any better. The speed drop was about 23%, and that's 2.42fps for lanczos'ed Plane clip. So much for postprocessing.

And finally, xvid encoding. I am doubt that anyone will manage to playback 720p xvid clip with bframes and qpels in realtime, but who cares :-)

Time was limited, so only "plane" clip was tested, more settings were used instead.

Real men always encode at quant2 of course, so did I.


ME6 Q2 H263 chr.ME+qpel,no lumamask: 1.38fps 3.531MB
ME6 Q2 H263 chr.ME+dyn.qpel,no luma: 1.39fps 3.531MB
ME6 Q2 H263 no goodies, w. lumamask: 1.58fps 3.522MB
ME5 Q2 H263 no goodies, no lumamask: 2.22fps 3.500MB
ME6 Q2 H263 no goodies, no lumamask: 2.06fps 3.481MB
ME6 Q2 H263 chr.ME+gmc, no lumamask: 1.80fps 3.384MB
ME6 Q2 H263 chroma ME, no lumamask: 1.89fps 3.367MB

ME6 Q5 H263 chr.ME+dyn.qpel,no luma: 1.32fps 1.159MB
ME6 Q5 H263 no goodies, w. lumamask: 1.53fps 1.154MB
ME6 Q5 H263 no goodies, no lumamask: 2.14fps 1.147MB
ME6 Q5 H263 chr.ME+gmc, no lumamask: 1.78fps 1.120MB
ME6 Q5 H263 chroma ME, no lumamask: 1.87fps 1.109MB


I have no idea why luma masking makes filesize bigger. I did a test long ago and it was shaving off about 1-2%, but today it wasn't working, no matter what clip i fed to it.

I didnt test b-frames, I dont get results I am supposed to with current build.

Chroma ME was a pleasant surprise. You lose some speed but i always prefer free compression over it. I couldnt see any difference with it enabled.

Also, high quantizers look great from some distance. Jeez, I couldn't see any diff. between quant2 and quant10 from 2 meters away on 19"... That will make battles with idiots that claim "my movie is half bitrate comparing to yours and it looks absolutely same" even harder.

And as you see the speed is about 2fps on p3-840. Now I want to see a setup that can do "1080i mpeg2->ivtc->720p resize->xvid" in realtime. p4-8Ghz anyone? I still think it's memory dependant, I wouldnt be surprised if somebody will report ~10fps with rdram p4-2's.

[Stereodude]

My I845PE powered P4-2.4b @ 2970Mhz pulls about 7.5fps using mpeg2dec3.dll and AVIsynth with a Lanczos resize to 720x480. This without using telecide and decimate.

I decode 1080i, seperate the fields and toss out every other frame (basically dropping the even fields), do a little cropping of the edges, pass to Vdub, resize to 720x480 (Lanczos) and compress to HuffYUV. I realize this isn't really comparable to what you're doing, but I figured I'd toss it out anyhow.

Stereodude

[The Belgain]

Right, I have a question which isn't specifically an HD question, but it becomes even more important here. Is there any way to use my graphics card's (Radeon 9000 Pro) hardware MPEG-2 decoding when encoding? It would make a huge difference to the speed for things like this.

Obviously things like resizing, filtering, encoding, etc would have to be done in softwarem but all the same it would save you a whole lot of time. Same goes for normal DVD ripping.

[OUTPinged_]

@The Belgian:

First, your videocard is supposed to decode to overlay if i recall correctly. It isnt so easy to pass that data to avisynth at a speed that would make it far faster than mpeg2dec3.

Also, as you can easily see from the benchmarks below, the speed drop after ivtc, filter and encode is 3.75 times and that means that even if you will half the decoding time, you will only gain a 13% increase in speed. That seems to be much, but it isnt, assuming that in a fastest setup it would be a 2.22->~2.5fps jump.

Also dont forget that all these benchmarks were tested using the most basic set of very well(i hope :-) optimised filters, and if you add something like c3d or smartsmoother, you will become old before you will finish filling that dvd with encoded content.

[The Belgain]

Ok, granted it may help less in that situation. What I was thinking of was using it when encoding to DVD-res MPEG2. Here you can just throw out half the fields, which is very fast and means you're dealing with non-HD material for all the resizing etc after that. Also CCE is reasonably quick. In this scenario the decoding could well account for over half the CPU cycles, so there would be a very significant speed increase.

[OUTPinged_]

the encode would look crappy :/

[The Belgain]

Why? It shouldn't really look much worse than a DVD (which despite the forum we're on, does look pretty good).

[trbarry]

If you have very sharp clear HDTV material I think throwing out half the fields can sometimes lead to encodes that are overly sharp or show aliasing artifacts a little bit. So a little vertical filtering might help in this case.

Of course for high quality HDTV it's tempting to keep it with more resolution anyway.

But for something like my upconverted Buffy caps it doesn't matter since the material is already soft and filtered for interlace so I do usually just pitch the odd lines.

- Tom

[Stereodude]

I've done a lot of HDTV MPEG-2 to DVD spec MPEG-2 file conversions. I always toss half the fields. I don't find the results to be overly sharp. I hate blurry files and blending.

Stereodude

[The Belgain]

Another question: if you're encoding to PAL spec DVD (ie 576 lines), isn't it a bad idea to upsclae from 1080/2 = 540 lines? I suppose it's probably preferable to deinterlacing though.

[Stereodude]

Originally posted by The Belgain
Another question: if you're encoding to PAL spec DVD (ie 576 lines), isn't it a bad idea to upsclae from 1080/2 = 540 lines? I suppose it's probably preferable to deinterlacing though. I can't speak to that belgain because I don't encode PAL. You could do a test and compare it if you are interested. Howeer, I would surmise from my experiences that I would be happper dropping half the fields and enlarging it slightly in the vertical direction.

Neither solution is optimal though.

Stereodude

[The Belgain]

Ok, another question: which Avisynth resize filter should I use for upsizing (as in from the number of lines would be upsized from 540 to 576)? Also, is there a filter which throws out half the pixels on each line? So that I could halve both vertical and horizontal res to 960*540 and then just resize that to 720*576?

Would the results be worse for the horizontal resizing than just resizing directly from 1920 to 720 (which would be much slower)?

[OUTPinged_]

what about SeparateFields.SelectOdd.BilinearResize(720,480)? Or ReduceBy2.BilinearResize(720,480)? No idea which one will be faster, but bilinear resize should help with material being too sharp.

[smok3]

Originally posted by Stereodude
I've done a lot of HDTV MPEG-2 to DVD spec MPEG-2 file conversions. I always toss half the fields. I don't find the results to be overly sharp. I hate blurry files and blending.

Stereodude 540 should be enough to fill the overscan of the typical pal tv set, so you need to find a way to add some black bars at the top&bottom (i mean in the case you want to avoid resizing).

edit: nm, i keep forgeting we r talking about 16:9...

[sjchmura]

What is the best way to throw out half the fields and then resize.

For example,

If I have a 1920x1080i source (Enterprise) and want to go to DVD.

Currently I
1. Crop the little noise at the edges to 1776x1000
2. use the decomb.dll to IVTC to 23.97 FPS
3. Resize to 720x480

How would throwing out 1/2 the fields help? I will just get a 1920x540 picture then correct? That still needs to be resized but now I have 1/2 the infomation to do the bilinear resize ???

I am confused - sorry.

[trbarry]

The fastest way if you are only keeping 480 lines is probably:

Crop(????)
SeparateFields()
SelectEven()
Decimate(cycle=5)
SimpleResize(720,480)

- Tom