switching to Xvid ;)

[kronik]

hey all, i've used everything from flask to sbc in the past 2 years and i've just made my first xvid encode using koepi's 04102002 codec.. i must say, i am very impressed with the quality of my first try. knowing that people will work on it is always a plus.
anyways, i'm encoding a 2h45min movie and i'm getting some noise around moving objects (people mainly). which settings would be able to help this or is it probably a bit too low of a bitrate?
thanks,
KroniK

[soujir0u]

Yeah, XviD seems to give a lot of mosquito noise... Are you using MPEG quantizers? Try using H263 and see if you like the results. Or try using the latest "unstable" builds, with Chroma Motion on. Try using a higher resolution, those noise appear more on lower resolution clips.

[kronik]

cool...will try h263 tonight, then maybe the "unstable builds"
don't really want to raise the resolution of 640X272 though ;)
thanks for the help.
KroniK

[bond]

XviD has a pretty sharp picture even with low bitrates but on the other hand there also is what is called mosquito noise, like soujir0u already said. I also have the impression that XviD gives a "fidgety" picture.

[iago]

@kronik

Hehe, welcome to the XviD board man! ;)

iago

[Shayne]

there are settings in the unstable buildings of xvid and there are many filters for asynth that can help you with this.

The ones you finial choose is as always ur preferences and likes

welcome to the open codec

[Didée]

kronik,

yep, welcome to the board and to XviD!

You gave no information on HOW you are things doing ATM.
Anyways, 2h45m is pretty long. Everything depends on many things, but I would *strongly* suggest to use a lower resolution! Perhaps around 576*240, but that's just a guess.
Don't make the failure to insist on (unrealistic) high resolutions. In my beginnings with this stuff, I thought that way, too. But my mind has dramatically changed since then. All the way down to 512*yyy is delivering very good results. And it is much, much better to have a little smaller, but artefact-free picture than a bigger one, that is full of artefacts.

And in case you're still using FlasK, please turn around: (almost) everyone here will give you a friendly and educational kick in the a** :) :)
I mean, if you don't know/do already, look into the AviSynth forum here. Please. Your encodings will improve a lot.

Regards

Didée

[sam_b]

Agree entirely with Didee (sorry no accent).

I find that Qpel improves the fidgety look a lot, especially at low resolutions, so try that with CM in one of the new builds. I would say that at your bitrate Convolution3D is very likely to help. Stops random noise from jittering too.

If you are using 640xX then you are going to have problems without B-frames at this bitrate. Larger resolutions are only realy practical with them in my experience.

There's some stuff in the minority report thread that might help. Let us know how it works.

EDIT: erm... I meant "minority report thread" not "minority report". But you all knew that, right?

[kronik]

thanks for all the replies and opinions. i've been doing lots of tests on the same movie with sbc and xvid over the past couple days and will continue until i get the quality i'm looking for. i'll take everything u guys said into consideration...thanks again
KroniK


oh, and Didée, i left that flask shit behind 3 months into my ripping. i agree with the kick in the ass part ;)

[soujir0u]

Oops, when I meant use a higher resolution, I meant that you should use a higher bitrate too. :D (Unless your original encode was already saturated with Quant 2s.) Open your first-pass stats file with Gordianknot or something, I usually aim for 60-65% (about Quant 2.9 average).

If the original source has some mosquito noise, the result is much worse in XviD, so better get rid of those noise first. Try using Undot after resizing in Avisynth. Also, temporalsoften(1,5,0) works quite well (good when using MPEG quantizers).

[HarryM]

Q-pel needs higher compressibility!!!

At 65-75% is O.K.

[Didée]

I did no serious compares/researches on the qpel feature lately. Furthermore, I am speaking out of pure theory now, but anyway:

In principle, qpel should deliver better compressability, because the ME can "point" more precisly on the destination block, and thus lower quants are needed, resp. the compression gets better for a given quant.
The problem is (was), that with activated qpel the search distance is cut to half, and therefore the ME might not find the best possible destination block. This must obviously lead to worse compressability in mid/hi-motion scenes.
Now, this problem should have become minor since XviD has dynamic qpel decision in the latest alphas: using qpel on lo motion, and take the benefit. On mid/hi motion, take classic hpel to catch the motion better.

If the above is not right, *please* do correct me!
I'm not into the codec's inner mysteries (as you might see from my probably not-quite-right terminology above) ... but in general, I like theoretic approaches, and I (would) like to understand how things are really working!

Regards and Thx

Didée

[sysKin]

Originally posted by Didée
In principle, qpel should deliver better compressability, because the ME can "point" more precisly on the destination block, and thus lower quants are needed, resp. the compression gets better for a given quant.
The problem is (was), that with activated qpel the search distance is cut to half, and therefore the ME might not find the best possible destination block. This must obviously lead to worse compressability in mid/hi-motion scenes.
Now, this problem should have become minor since XviD has dynamic qpel decision in the latest alphas: using qpel on lo motion, and take the benefit. On mid/hi motion, take classic hpel to catch the motion better.

If the above is not right, *please* do correct me!
I'm not into the codec's inner mysteries (as you might see from my probably not-quite-right terminology above) ... but in general, I like theoretic approaches, and I (would) like to understand how things are really working! OK, since you asked :)

Qpel does not limit the range. For any frame, encoder makes a decision - how big range of motion vectors it needs. There is a value, called 'fixed code' (or fcode, iFcode in xvid sources).
If qpel is active, encoder just chooses a bigger range - when fcode is increased by one, the range is doubled.

However, when a vector is written to the bitstream, for every of it's components (x, y) a number of bits equal to fcode is written, followed by a VLC corresponding to the vector. If the range is doubled, two more bits (statistically) are needed for every vector.

If we assume that there is one vector for every macroblock (there might be 4 or 0), at the resolution of 640x272 and 24fps and P-frames only, two bits for every macroblock take 32.5 kbps.

Qpel generally decreases compressability. Although it can find a better match, it usually doesn't 'correct' this match anyway, because the errors are too small. If there is no good match, qpel doesn't help much either - the amound of texture data is still similar.

I hope this helps,
Radek

[Didée]

[smoke coming out of my head]

Thanks very much for clarification, sysKin.

I would lie if I'd say I understand it now completely ;)

However, this was a pretty good example what silly assumptions sometimes may grow!
I took all the little hints that were dropped here, there and elsewhere by people with insight into XviD's technology, and tried to puzzle them together in a logical manner.
The result was crap.

But, sorry for being penetrant:

> If qpel is active, encoder just chooses a bigger range [of motion vectors]

>Qpel generally decreases compressability. Although it can find a
>better match, it usually doesn't 'correct' this match anyway,
>because the errors are too small. If there is no good match, qpel
>doesn't help much either - the amound of texture data is still similar

Reading that, I start even more wondering:
- Range is increased by qpel ???
- What IS the benefit of qpel? From your explanation, I understood that it OTOH decreases compressability, and OTOH doesn't help much ???

I thought it has to do with sub-pixel accuracy?

Pretty :confused: now

Didée

[Koepi]

hpel and qpel _are_ sub pixel resolution.

The benefit from it is the more precise movement detection - thus possibly saving texture bits, which get consumed by bigger motion vectors usually.

The error that this more precise detection produces is smaller, that's why it's considered to improve something(the amount of error correction data gets less). You can see the success of it in the sharper image you get, it has more details - even if you have to compress the material more.

The search range is still the same:

if in fullpel 32 pixel range is searched, this happens in hpel and qpel as well (hm. not sure about the hpel part). So the search efforts increase (think of it as if you'd blow up the image by the factor of 4 in each direction).

I hope i didn't cause more confusion now %)

Best regards
Koepi

[Didée]

No, Koepi, no more confusion. The Mud is clearing, the puzzles start to form a motive ...

So I see the idea I had wasn't totally wrong, only half-way (or so).

Thanks

Didée

[kronik]

so, qpel on lo mo and hpel on mid-hi mo? no one agreed or disagreed with Didée on that :)
thanks
KroniK

[Teegedeck]

Not really. Quarterpel can make it harder to fit a movie on one CD. Nothing to do with high motion. (Theoretically?)

You know that all unstable features aren't recommended for archieving purposes?:o

[kronik]

i don't make 1 cd rips ;)

[sysKin]

Originally posted by Didée
Reading that, I start even more wondering:
- Range is increased by qpel ???Not by qpel. Codec makes a decision about range, for every frame. If for a given picture it needs, say, 32 pixels, it chooses 64 halfpixels when in halfpel mode (fcode 2) or 128 quarterpixels in qpel mode (fcode 3).- What IS the benefit of qpel? From your explanation, I understood that it OTOH decreases compressability, and OTOH doesn't help much ???
Most macroblocks in mpeg-4 consist of motion vector alone - with no additional texture data. In qpel mode, those macroblocks will look much better, because they will be choosen with better precision.
It doesn't mean that macroblocks with additional texture data (to fix a not-so-good motion estimation) won't benefit - they will also look better, and they _might_ have less texture data - but mostly they will just look better.

Ask more questions, I'm pretty sure you're still confused, as I don't explain as clearly as I'd want to ;)

Radek

[kronik]

i tried it last night and my computer must have restarted while i was sleeping :( going to try again tonight.

@sysKin
thanks for clearing that up for me. i understood what qpel and hpel is now :D
thanks

[kronik]

i did a rip with these settings and got wierd motion problems. when the opening credits came up on picture or black background the blocks seemed to be jumping around. it doesn't seem to do it during a preview in vdub. here's the settings i used, any ideas?

2 Pass Int.

Bicubic 0.60
584x248

Motion search precision: 6
Quantization type: MPEG
FourCC: XVID
Max I-frame Int: 260
Min I-frame Int: 10
Qpel: enabled
Chroma motion: enabled
Max B-frames: 3
B-frame Quant ratio: 150
B-frame Quant offset: 100
DX50 B-VOP compatibility

Min I-frame Quant: 2
Max I-frame Quant: 6
Min P-frame Quant: 2
Max P-frame Quant: 12

I-frame boost: 0
Discard first pass: enabled
Below I-frame distance: 10
I-frame bitrate reduction: 30
High bitrate scenes: 0
Low bitrate scenes: 0
Bitrate payback delay: 250
Payback proportionally

Method: Smartripper, DVD2AVI, Vfapi plugin, Vdub.
thanks
KroniK

[MaTTeR]

Originally posted by kronik
, Vfapi plugin, Vdub. Sorry to jump in but the word "VFAPI" caught my attention*grin* Just curious if you have a specific reason to use VFAPI. The Vdub/AVS combo will provide much better results instead of VFAPI

[kronik]

thanks for the advice, i'm getting into that as soon as i can work out my bugs with xvid encoding :)

[crusty]

Ok here goes guys....

After re-reading this thread for about six times I think I got it now.
I'll try to put what I think you said in a bit more n00B-friendly words.
If I'm wrong anywhere don't hesitate to correct me ok?

Didee:
The problem is (was), that with activated qpel the search distance is cut to half, and therefore the ME might not find the best possible destination block. This must obviously lead to worse compressability in mid/hi-motion scenes.
SysKin:
Qpel does not limit the range. For any frame, encoder makes a decision - how big range of motion vectors it needs. There is a value, called 'fixed code' (or fcode, iFcode in xvid sources).Qpel generally decreases compressability. Although it can find a better match
Koepi:
hpel and qpel _are_ sub pixel resolution. The benefit from it is the more precise movement detection - thus possibly saving texture bits.The error that this more precise detection produces is smaller, that's why it's considered to improve something

- Range is increased by qpel ??? Not by qpel. Codec makes a decision about range, for every frame. If for a given picture it needs, say, 32 pixels, it chooses 64 halfpixels when in halfpel mode (fcode 2) or 128 quarterpixels in qpel mode (fcode 3).

Most macroblocks in mpeg-4 consist of motion vector alone - with no additional texture data. In qpel mode, those macroblocks will look much better, because they will be choosen with better precision.


Ok, here's the deal:

Mpeg4 compresses video by the use of macroblocks.
Say you have a movie with a red car driving to the right.
In Mpeg4, the image of that car will be made up by macroblocks that contain the texture of the car (i.e. in this case they will be all red).
When the car moves a bit to the right in the next frame, it won't change the content of the textures that make up the car, just their position (we assume here there is no noise). So in the next frame the macroblocks with textures in them are replaced by motion vector macroblocks.
These are a sort of placeholders that tell the codec that the corresponding texture macroblock in the previous frame moves amount x to the right and amount y upwards.

Now this motion has a certain amount of precision.

Say you have a macroblock moving 1 pixel to the right and 1 pixel up,
then the movement is by a whole pixel and you do not need subpixel precision to correctly contain the motion inside the motion vector macroblock. Right?

Now if you have a macroblock moving 1 pixel to the right and half a pixel up, you need Half-pixel precision to correctly contain the motion, because otherwise the motion will be rounded, in this case probably up, and you will end up with the same motion as in the previous example.
However, this would lead to an incorrect placing of the texture macroblock in the next frame.
Unless we tell the codec to use Half-pixel precision the codec will have to adjust the texture inside the badly placed texture macroblock and therefore use texture bits.
So basically the error forces you to use more bits for textures, thereby reducing quality at an equal filesize.

Now say we have a macroblock that moves half a pixel to the right and one fifth (0.20) pixel upwards. With halfpixel precision, the macroblock will move half a pixel to the right, but 0.20 will get rounded down to 0 and the macroblock will not move up at all.
Again, we get a motion error that has to be compensated for by texture bits.
Quarter-pixel precision will, in this case, still get not entirely accurate motion, but precision will be much higher.

So basically, it is not range of motion searching that is increased by using Hpel or Qpel, it is the RESOLUTION at which we search which is increased.
As an analogy:
Full-pixel precision is like looking at the movement of an ant with our bare eyes. You can see if he's going to the right pretty good, but that's about it.

Half-pixel precision is like looking at the ant with a magnifying glass. If you don't fry him in the sun you can see that he's walking to the right but also slightly of upwards.

And with Quarter-pixel precision you're using a microscope to look at the ants movements. You can actually count his steps and see that for every 5 steps he makes to the right he's also making one step upwards.

So whatever you do, Qpel will give a more precise motion to your encodes, and if you don't have to worry about bitrate it absolutely WILL INCREASE QUALITY.
Now offcourse everything in life comes with it's price and so does the higher precision.
I know I'm probably not portraying this correctly, but bear with me.
The higher search precision won't do anything if it isn't contained in the encoded frame. So every motion vector macroblock has to use additional bits to properly contain the higher precision.
As said by SysKin, that could amount to quite a significant percentage of your entire bitrate. So you're paying the Qpel option extra motion vector bits in the hope that you will save a higher amount of texture bits.

So at a given filesize, Qpel will make your encode look better if the amount of texture bits saved by the higher precision is bigger than the amount of bits used up by Qpel itself.

But, if the amount of texture bits saved is smaller than the amount used up by the Qpel option, the gain in quality by Qpel will be smaller than the loss in quality caused by the loss of bits (to the Qpel option).

Offcourse there is NO WAY TO PREDICT THIS, unless you manually look at all the motion vectors in all the frames of your video and then somehow statistically decide wheter or not Qpel would give you a better looking encode. Which is offcourse totally inpractical.

Out of this follow a few things:

1)
mf told me that SysKin reported he got better results using Qpel with sharper clips than with softer clips. Now that we've made some sense out of the Qpel option this actually makes sense:
With softer clips the differences between edge pixels on adjacent macroblocks will be smaller than with sharper clips. So if Hpel makes an error, it will be easier noticeable in sharper clips than in softer clips and more texture bits will have to be used to compensate.
Qpel will make less noticeable errors than hpel in sharper clips and will therefore save more texture bits. Thus, in sharper clips, Qpel will save more bits that would otherwise have gone to texture correction than it does in softer clips. So Qpel is bound to have a bigger effect on sharper clips.
As a side effect, if Qpel implementations in Divx and Xvid are equally efficient, xvids Qpel would still be more beneficial because xvid is sharper by nature.

2)
Qpel WILL increase quality! If you have undersized files ALWAYS turn the Qpel option on.

3)
Kronik:
so, qpel on lo mo and hpel on mid-hi mo? no one agreed or disagreed with Didée on that
Basically your question has no answer. It completely depends on the precision NEEDED by the motion vectors, not on the SIZE of them.
So a low motion movie could either benefit from using Qpel or you could simply be wasting bits. There's NO WAY TO TELL BEFOREHAND.
You simply have to encode it to see.

4)
Since the added precision off these motion vectors are contained in the encoded clip, I guess one could build a small program to check how many bits are actually used up by using Qpel. And I'm not sure, but maybe it could perform calculations on the added precision to see if it really DOES save enough texture bits to make it worthwhile.
That way, you would still have to encode clips both with and without Qpel, but then you would have an ABSOLUTE measurement of wheter or not it's worth the effort.
-----------------------------------------------------------------


Well that's basically the idea. If I'm wrong here please correct me at your best convenience, but if I'm right somewhere please tell me as well! That's a special request to all the developers, cause if this is a proper explanation it could help people a lot.
Thanks in advance.:)

PS:
I actually wrote something like this this afternoon but when I wanted to submit the forum was down and I lost the entire post....ARGH!!:( :(
This time I made a backup..:D

[PerCIVaL]

Thanks for the layman-friendly explanation crusty, I'm quite new at all of this so I beg your pardon for the stupid question I'm about to ask.

How is it possible for a macroblock to move a fifth of a pixel? If I understand correctly an image consists of full pixels right? Doesn't my screen display in full pixels as wel?

I feel stupid for just asking but this was bugging me for a while now.

[PerCIVaL]

wait a second!

My brain suddenly told me this :p

these are all vectors in a virtual 2d field right? I could understand how one could represent a real life image (or analogue capture) in a 2d field with half/quarter pixels.

But doesn't, for example, a dvd already output images in fullpixel only? How can a macroblock move a quarter pixel one frame to another then?

how big are these pixels anyhow? Was there more than one shooter at the jfk assassination?

am I making any sense? :confused:

[Koepi]

There you go crusty: make the explanation easier by repeating everything over and over again *g* didn't help it.

percival: a vector is about _directions_ and _power_, not about pixels (AND: we are talking about movement of macroblocks of 16x16 or 8x8 pixels). Think of a snippet from an eclipse or something like this:


*-\
\
\-*

with the stars being start- and end-point.

Regards
Koepi

[crusty]

Well it's not the macroblock that's moving with sub-pixel precision, but it's the original footage that's moving. It's the codecs attempt to compensate for this by making roundings more precise.

Let's say for simplicity that a pixel of the output is 1 meter large.

The original footage , the input, moves 20 cm in the next frame.
It could do that easily if it was 5 times the output resolution, or if a pixel in the input was only 20 cm.
That's one fifth of a pixel of the encoded output.
But the codec can't display that properly because it can only display pixels.
So the codec has to do some rounding.
Both with full pixel and half pixel precision the rounding would be down to: Not moving at all. So the codec doesn't do anything. It just says in the next frame in the motion vector macroblock: No movement.
But with quarter pixel precision it gets rounded up to one quarter movement. So instead of movement being 20 cm of, it's just 5 cm of. So it's more precise.
Offcourse the same is true if the movement was 1 m 20 cm or 2m 80 cm, or anything like that.
But because instead of having to put in the encoded file that the macroblock moved either 0, or 1 or 3 meters, it now has to put in the encoded file that the macroblock moved 0.25 meters, or 1.25 meters, or 2.75 meters. So that extra movement information needs to be stored and that needs more bits.
So I guess the Qpel is done on the input to position the output (the macroblocks) more precisely. Get it?

Mind you, I still haven't gotten any confirmation of the developers if my explanation is actually correct. :D
So unless they say so, don't consider my explanation as a holy thruth or anything.. :p

[crusty]

Was there more than one shooter at the jfk assassination?

Yes. Didn't you see (or rip) JFK? :D :D :D

@Koepi:

Was my explanation actually correct?

How about the conclusions?

If true, at least the fact that we can turn it on with undersized files for a real increase in quality would be a real benefit.

[Koepi]

crusty: the MB is moving with subpixel precision as well ;) Else everything is correct as in: as it can be without getting technical.

EDIT: you edited that last post when I couldn't see it, but the filesize assumption isn't something I can backup. Of course you can switch it on, but it doesn't necessarily increase your filesize. Depending on the source it will _decrease_ it (remember the better texture match).

Regards
Koepi

[crusty]

So Koepi, does this mean that:

A encode with resolution 512 x 300, that would be displayed at 2048 x 1536 would be more accurately displaying movement?
Because, a macroblock that was 8x8 in the original resolution, would now be 32x32, and therefore, a movement of a quarter pixel in the encode, would now mean a movement of an entire pixel while being displayed?

So if the macroblocks also move with subpixel precision, then playing them at such an enormous resolution would have the advantage of getting much more accurate movement, right ?


Koepi:
Else everything is correct as in: as it can be without getting technical.
Kewl! I got it right ! :D
Thanks for checking man! Pfew....


ALSO EDIT: Yes, but I didn't mean that filesize would increase, just that quality would increase.
Filesize could actually decrease, which I find just an added bonus, because you'd have more room for audio.:cool:

[Koepi]

The image gets played back in it's native resolution. When watching at bigger resolution it simply gets upscaled and filtered (hopefully. Ever seen an unfiltered, upscaled image?).

The point in subpixel precision is to get a better match in the "original" texture so that the error compensation needs less space and the original image gets represented better (like in: PSNR). (I feel like I lost the ability to translate techtalk into real life talk...)

Ok, I need a nap now,

Good night everyone,

best regards
Koepi

[Tommy Carrot]

Theoretically, this is all true, but don't forget, qpel has the bad habit to amplify the DCT rounding error, so the image of the encoding becomes "dirty", or noisy (is this the unfamous "smearing"?). Usually, this is almost unnoticable, and the benefit of the qpel is larger than the drawback. But other times it can be really annoying. So i wouldn't use qpel for high quality encoding.

I think, this is the reason that h.264 uses Hadamard transform instead of DCT. Theoretically, DCT is better, but there are always rounding errors.

[crusty]

Theoretically, this is all true, but don't forget, qpel has the bad habit to amplify the DCT rounding error, so the image of the encoding becomes "dirty", or noisy (is this the unfamous "smearing"?).

Well, if somebody could explain to me what the DCT rounding error is, and what DCT actually is,( i have only a vague idea of what it is really), I could then edit my explanation to explain this in a clear way to less experienced users.

And is the infamous smearing actually a bug or is it an inherent risk of using Qpel ? That's offcourse mighty important for people considering using Qpel.

[Tommy Carrot]

Well, divx has got the same effect with qpel, so i think, it's a general issue with qpel. So bad, because qpel can be very powerful tool.

DCT is the base of all jpeg/mpeg compression.

[unmei]

hmm i can't explain DCT because i do not understand enough of it, but it's a "basic" mathematical algorithm, not limited to video stuff. Im sure you can get tons of explanations and discussions on science sites like scirus, the full name is "Discrete Cosine Transform" - or if you're in a university your algebra prof will most likely love you for asking him :p


this might give a short intro, the DCT is about in the middle of the page: http://www.jmcgowan.com/avialgo.html

[chemmajik]

Just wanted to say to all the XVid developers I pestered in the beginning like crazy that all of you have done a fine job, especially with the capture 1 pass routines. I havent really had time to deal with multipasses as much, but 1 pass CBR looks pretty good for my high bit rate 2300 captures. Having a faster processor helped me when I upgraded but its like the original viewed, plus it scales up better then any codec I have ever used. Lowres ntsc captures can look near the original great for 1gig file saves per show or to view later when you have time its almost near TIVO with VirtualVCR or VirtualDubVCR... and definently takes up a heck of alot of less space then using PowerVCR Mpeg2... which is a b if you decide to archive with limited to below 5gigs hd space at times.

[duartix]

Why capture at CBR?
This is something I really don't understand!
Why not capture at constant Quality? Or constant Quantizer? Like Q3 or Q4 if you don't have the space?

[Jehovantodistaja]

Why isn't kronik banned for using a LOGO of an ILLEGAL DRUG on his avatar :confused:

[Wormz]

Originally posted by Jehovantodistaja
Why isn't kronik banned for using a LOGO of an ILLEGAL DRUG on his avatar :confused:

Marijuana is not illegal everywhere, you know...

[Jehovantodistaja]

Originally posted by Wormz
Marijuana is not illegal everywhere, you know...

warez neither

[Wormz]

Originally posted by Jehovantodistaja
warez neither

I think this has gotten enough off-topic.

The point is, forums can be closed down for having warez links and the such. If you know of a forum that was closed down for having images of marijuana, let me know.