Forgetful developer - need suggestions

[-h]

If anyone's suggested some XviD feature / bug fix, and I've said I'll do it, odds are I've forgotten. If people can post their ideas to this thread, I'll edit their posts with a [DONE] if I get it merged.

-h

[kastro68]

Hey,

Could you fix the multiple of 16 resolution requirement?

Thx

[serbersan]

It's fixed the problem with luma masking in first pass? I mean the idea about disable it always for first pass. I downloaded Nic 22 build and it hasn't been solved.

Maybe this was a silly thing, but it's posible to use some kind of ToolTips like for example in java. When you put the mouse over an option appears a little menu with an explanation, advice or warning of the switch.

Has sense to make profiles like:
Using normal 2 pass encode with luma masking in 2 pass
Using normal 2 pass encode without luma masking and using restrictions for I-frames, etc..
....

I've read the way that is very experimental about what quantizer to use H.263 or Mpeg depending of some values obtained in the first pass or 2. What about some kind of compressibily check to know it.

Only my 2 cent, I hope it can help.


BTW I've found a little bug in 22 nic build with the final size. The desired filesize was 1212 Mb and the final 1208. Used the defualt options of the encoder H.263, Luma Masking only in 2º pass, no credits used.

[Ripe73]

BTW I've found a little bug in 22 nic build with the final size. The desired filesize was 1212 Mb and the final 1208

Did you calculate the bitrate with GKnot?
If you did dont forget to disable the "Calculate Frame Overhead" maybe thats the 3MB you talking about.

[rui]

Originally posted by Ripe73


Did you calculate the bitrate with GKnot?
If you did dont forget to disable the "Calculate Frame Overhead" maybe thats the 3MB you talking about.

I don't know about Serbersan, but I sure did forgot :rolleyes:
I have been letting Gnot calculate frame overhead, but since I do the audio part separately, in Gnot I just have to point the audio file size and then copy the video size it gives me to XviD second pass configuration, right?
But what about in the audio tab, do I still have to enable 1x vbr mp3? Or just put no audio, since I already loaded the audio file size in the Audio A tab?

To -h:

I would like to see a Modulated quantization tab supported by the codec without having to rely on Koepi's changing it, in spite of having full confidence in Koepi's work. ;)
It just would be, lets say, "official" :p

And I would like to have post processing, tired of using divx4 post processing (still usefull when doing 1 cd rips)

[Dali Lama]

To The Developers (-h):

I have never heard of a codec being designed for encoding cartoons/anime, but here is one problem that seems to be occuring across all codecs. It involves encoding anime and here are some recent quotes...

"the only problem XviD has with animation is more artifacts around sharp edges." -Western Shinma

"I have been testing Xvid on aime as well, but I get motion trials." -Aktan

In comparison of Xvid to Divx 3.11 (Nandub) Manono writes, "If that's the only motion artifacting you can find using XVid, then I'd say you're a lucky man, and maybe it's time for me to switch."

This final comment suggests that if some tweaking to the codec could be done to be optimized for anime/cartoon (even as a separate addon codec or something) then Xvid could take the market as the best codec for this purpose. Which btw many people use it for.

As far as how this could be done, I was thinking of putting some extra type of detection to find sharp edges in anime and give them more bits. Use another detection code to find static backgrounds and give them a stable bitstream with even a slight blurring effect.

I hope this sounds intelligent, as I don't know how to make a codec.

Ok, Thanks goes to everyone contributing to Xvids development,

Dali

[serbersan]

I enter in XviD desired size 1241088 and I obtained 1236992 it there's no relation with gknot.

BTW the date of the compile of Nic is 21 but in the page it puts 22 I believe

[Koepi]

An error of 50 kb isn't worth discussing about, that can be easily the thing that your source isn't that compressable.
If you tweak with the quantizer locking mechanisms this can happen.

Still, 50k at 1.2mb is absolutely in range.

Regards,
Koepi

[philippas]

The problem with merging parts of Xvid together. For example, if you trim the credits and try to append them afterwards it doesn't work.
It's not a big issue just now but in the short future if you want to merge say 2 cd rips and put them in a DvD-Rom you will have a problem !

[kastro68]

Just keep it as 2 separate tracks

[philippas]

@kastro68 What do you mean ? To seperate tracks in the DvD ?

[kastro68]

I'm not too familiar with DVD-Ram atm,

but i always thought that you can add an extra chapter for the credits. That way you can skip straight to the credits to check the name of the chick in the movie. Well this would be the only reason I would keep the credits.

Ja ne

[meleth]

Originally posted by kastro68
I'm not too familiar with DVD-Ram atm,



but i always thought that you can add an extra chapter for the credits. That way you can skip straight to the credits to check the name of the chick in the movie. Well this would be the only reason I would keep the credits.



Ja ne

haha exactly the only thing i use the credits for:P Well occationally there might be a good soundtrack that i wanna see who has made.

[Selur]

may be: something like a partial temporal smoother would be cool/usefull,...
I mean a smoother filter that will only kick in if the quantizer goes over xy. Not too sure if this might result in funny results, but I think it could be cool in action intensive movies, loose some sharpness in action intensive sceenes instead of getting artefacts,...
(just a thought)

Cu Selur

[meleth]

Ok, i haven't tested xvid myself yet so this migth already be in. However, wouldn't there be alot to gain in speed if some of the most common filters found their way into the codec? I'm thinking resizing,deinterlace,crop,ivtc here. This should ofcourse be made so the "filters" can get upgraded/replaced without the need to release a new codec.

Shouldn't this also make it possible to make even better filters since people will be able to access both the original frame and the encoded frame. Or is this possible already?

[kastro68]

I thought that after playing one chapter the dvd would automatically jump to the next chapter. With my very limited knowledge, I don't think there really is a need to merge the files.

But then again, did you want to convert the movie back into dvd format or did you just want to store it on a dvd-rom in Xvid format?

I am guessing the latter. If it is the latter, then I apologise for misunderstanding the situation.

I haven't tried merging 2 Xvid clips together yet, but i have tried merging bitmap sequences to divx 4.12. In the process I have learnt that it is very hard to merge video files because it must have the same video sample size, same resolution, same framerate and same audio bitrate. Are you using VBR audio, if so i assume this would be the reason why you are unable to merge the 2 clips.

[-h]

OK, here's my todo list, from the looks of things:

- Resolution problems
- Tooltips (I like this)
- Disabling more controls dependent on mode
- File merging problems (haven't tested this, might be a VBR audio problem?)
- Minimum frames between I-frames
- Finally commit Foxer's curve compression (very nice)

These will have to wait:

- Modulated quantization support - this is waiting on "other" core mods
- Cartoon mode - substantial modifications required, though Isibaar has already done some work in this area
- Internal smoothing - Isibaar is looking at this
- Internal filters - speed increase is minimal, apart from certain special cases: deinterlacing "works" faster, but is poorly implemented in DivX4, and should be dumped in favour of "real" MPEG4 interlaced frame support. Other filters will see minimal speed increases.

More suggestions are welcome.

-h

[Nic]

(Just on a Minor note:
I implemented the DivX4a50 deblocking & deringing (in C)...I might have implemented it slightly wrong, but the results weren't as good as expected (however the deringing was quite good really).....(there were also obvious bugs in the divx4a50 code)

(Im trying to add one of the deblocking algorithms proposed at mplayerhq...ill keep U posted)

-Nic

[Selur]

about the merging problem, if i remember right only aviutil was able to merge 2 Xvid files,...

Cu Selur

[meleth]

Originally posted by -h



[



- Internal filters - speed increase is minimal, apart from certain special cases: deinterlacing "works" faster, but is poorly implemented in DivX4, and should be dumped in favour of "real" MPEG4 interlaced frame support. Other filters will see minimal speed increases.









-h [/B]







I don't mean that you have to develop all these filters yourself. It would probably be better if you could just supply an api that filter developers could use?



Ofcourse doing this would probably need some input form the filter developers out there as to what they'll need access to.

[-h]

I don't mean that you have to develop all these filters yourself. It would probably be better if you could just supply an api that filter developers could use?

It's possible, but the problem is you'd see a very very very small speed increase, if any at all, and add a heap of complexity to the codec while you were at it.

Actually, if you wanted to follow this through, you'd be much better off writing a shell for XviD which allowed it to act as an AVISynth plugin. It'd take some work, but that's really the most "correct" method of integrating the codec into the filter stream. Well it'd take some AVISynth mods too, since it's only really designed for raw frame output.

@Nic - the DivXa50 post-processing code looked incredibly ugly, any ideas whether ffmpeg / mplayer are better implementations?

-h

[kastro68]

I just joined 2 Xvid files (Without the audio) and it worked fine.

I used Virtual Dub's append feature.

ja ne

[meleth]

Originally posted by -h
Actually, if you wanted to follow this through, you'd be much better off writing a shell for XviD which allowed it to act as an AVISynth plugin. It'd take some work, but that's really the most "correct" method of integrating the codec into the filter stream. Well it'd take some AVISynth mods too, since it's only really designed for raw frame output.


-h [/B]

mmm yeah, that would probably be the best way.

Oh and while i'm at it. And slap me if i'm smoking crack. But i've been doing some thinking. This applies to the cartoon mode. Since most anime is originally 20 or 15 fps that they then blend to get a higher fps ans smoother playback. Would it be possible if all the blended frames would get dropped, and then have the xvid DS filter once it's completed perform this blend instead? The quality of the encodes should get significally better while still getting smaller file sizes.

[serbersan]

Sorry for my ignorance but, What kind of artifact is ringing.

Some explanations about other types of artifacts?

I only know about mosquito noise and blocking.

Thanks.

[Nic]

The mplayer algorithm is much cleaner, instead of using the N-Tap (9tap in the case of divx4) it uses adaptive filtering to do false edge detection then apply smoothing using a very simple alogorithm. The effect is impressive (although might not be quite as good as MPEG-4) However, the processing for the mplayer strong filter is alot less.

e.g.
if the mpeg-4 algoritm needs 2184 additions, then the mplayer algo needs only 302(!)

So ill implement that soon & see if the results are any better....
....The deringing filter of DivX4a50 was quite good, I might keep that
however, it needs to be optimised (not one of my strong points :)

Anyway ill keep you posted :)

-Nic

[kastro68]

@Serbesan

Ringing is most obvious on cartoon encodes. I guess the best way I know to describe it would be mosquitos near lines.

[-h]

Oh and while i'm at it. And slap me if i'm smoking crack. But i've been doing some thinking. This applies to the cartoon mode. Since most anime is originally 20 or 15 fps that they then blend to get a higher fps ans smoother playback. Would it be possible if all the blended frames would get dropped, and then have the xvid DS filter once it's completed perform this blend instead? The quality of the encodes should get significally better while still getting smaller file sizes.

This would be possible (while being incredibly slow to encode), but would unfortunately break compatibility with MPEG4. There are many ways MPEG4 could be improved, but it makes more sense to focus on a spec-compatible codec, as we'll have lots of fun devices to play it on sooner or later.

Well actually, B-frames could achieve this very nicely. Depends on what gruel eventually produces.

-h

[Nic]

Yes mosquito lines is one way of describing it :)

The ringing & mosquito noise/halo effect are the same thing really (or at least to my knowledge :) and are tackled the same way...) (i.e. using an adaptive low pass filter to avoid destruction of high frequency details)

I might also change this...the drop in PSNR is minimal, but the image quality can be greatly improved...The deringing filter of DivX4 is everso slow.

-Nic

[Selur]

another little suggestion: about the start&endcredit settings,..I really love these :cool:, but could u plz make it possible to let one separatly set the credits quality rate of end and starts credits,.. ?
(just, a little something that popped into my mind,...if it's not just a little, minor code change forget it)

thx for considering,..

Cu Selur

[Ripe73]

Requests

*Crispness Modulation

*High/Lowpass for bitrate

*Brightness scale on the DSfilter like 0-100

Thats all:)

[philippas]

1. Apply a smoothing filter when luma masking is enabled so that the picture looks less blockier when bits are taken off. And also a smoothing filter for very high quant values(i.e for the credits)

2. Able to join parts of Xvid. I tried virtualdub, aviutl 096&097, and i couldn't join parts of an Xvid encode.

[NeVeRLiFt]

Originally posted by -h
[i]
There are many ways MPEG4 could be improved, but it makes more sense to focus on a spec-compatible codec, as we'll have lots of fun devices to play it on sooner or later.
-h


:D
Thank you !!!!

[kastro68]

If it is not too much trouble could you make the Xvid DSF remember the last setting used.

Call me lazy, but when you watch 20 min anime episodes, you don't want to have to put the brightness up a notch each and every time.


Thx

[Nic]

LoL..... :)

Thats a good idea....Dont worry over the next week the DSF will look far smarter & work alot better.

-Nic

[gruel]

Originally posted by kastro68
Hey,

Could you fix the multiple of 16 resolution requirement?

Thx

:(

Actually no, we can't.

This is because of internal YUV format, which subsamples chroma. If the resolution is not multiples of 16, then chroma cannot be split cleanly into blocks of 8x8.

Only fix would be to copy the input image into a new bigger buffer and encode that. Then the output size is a multiple of 16 again, but there is a small black border than is difficult to encode. The result would be than the encoder gets slower and of worse quality for non multiples of 16. Not good either...

[-h]

Only fix would be to copy the input image into a new bigger buffer and encode that. Then the output size is a multiple of 16 again, but there is a small black border than is difficult to encode. The result would be than the encoder gets slower and of worse quality for non multiples of 16. Not good either...

This is already done in encoder.c, ne? The image is padded out to be a multiple of 16, and the "padded" mblocks are filled out by repeating the last valid sample value in image.c

If you encode a few test clips with xvid, whose resolutions are multiples of 16, 8, 4 and 2 (320x240, 328x248, 324x244, 322x242), this ends up happening:

XviD decoder plays 16 and 8 content correctly, 4 appears to have misreported stride, and 2 just seems broken.
DivX decoder plays 16, 8 AND 4 content correctly, 2 seems broken.

I was going to look into this further, but am waiting on the core mods..

-h

[gruel]

Originally posted by -h
If you encode a few test clips with xvid, whose resolutions are multiples of 16, 8, 4 and 2 (320x240, 328x248, 324x244, 322x242), this ends up happening:

XviD decoder plays 16 and 8 content correctly, 4 appears to have misreported stride, and 2 just seems broken.
DivX decoder plays 16, 8 AND 4 content correctly, 2 seems broken.
-h [/B]

Possible. But then it's just a fix for something that should not be done. Old rule in software development: as soon as something that should not be used isimplementent, every user will use it and complain that it's bad. :-(

[-h]

I remember reading about this in the spec doc, guess I'll have another look (might have only been about non-rectangular objects) around 3.3.1. But it did seem that non-multiple-of-16-resolutions were expressly catered for.

-h

[Selur]

Is it possible to optimize Xvid for DualCPUs ? And if so, are there any plans to optimize it ?

Cu Selur

[gruel]

Originally posted by Selur
Is it possible to optimize Xvid for DualCPUs ? And if so, are there any plans to optimize it ?

Cu Selur
Yes, actually I had a working version of multithreaded motion estimation for OpenDivX, but did not take it over to XviD. But since I have a Dual Athlon (and bought it exactly for that reason), I will do it one day.

However, it's quite a lot of work and maybe it's better to let the structure settle a little before adding SMP overhead. Maybe it'll go into a seperate branch, or will be based on #ifdef/MACROs. My favourite is function pointers that are set during runtime. But I'm not sure, yet. I don't even know if to multithread every routine seperately, or to multithread the whole encoder. Both has advantages and disadvantages...

Also since there are many other tasks in video encoding than just the codec (source decoding, audio, resizing) I nevertheless get 160-180% CPU usage DVD->XviD with Linux using transcode. (so 80-90% per CPU).

So the effect might be larger for 4 CPU machines than for dual.

[kastro68]

@gruel
this is a little irrelevant, but i would just like to know if you are running dual athlon mps or dual xps

thx

Ps: I am waiting for the dual abit boards to come out myself... so far only tyans and supermicro mobos are available in australia.

[Nic]

Ive noticed the function pointers set at runtime trick in the OpenDivX source.....erm....fun rofl :)

-Nic

[Logos]

I've accidentaly discovered that when the max KF interval is set to (for example) 300 and there is a real scene change at the 301th frame... the screen is really messed up: the first frame of the new scene is a DF. Could the min KF limit be ignored in such a situation ?

[-h]

The "idea" behind min keyframe interval is to prevent such keyframes.. I don't really like min keyframe intervals, but they do help with overflow management.

You can set the min interval to 1 to prevent this if you'd like.

If it still occurs when min=1, we have a bug..

-h

[Logos]

I thought that the min keyframe setting was to avoid having too many keyframes in heavy action scenes. My idea is to override the min keyframe setting *only* when the previous KF was created according to the max KF setting. Anyway, this particular case should be verry rare, so it's not a real problem.

BTW, I've tested the 20020302 build... it rocks :D Great Work !

[Logos]

I was wondering if motion compensation could be integrated in XviD (particularily for DVD rips).

[-h]

Motion compensation (or motion estimation, ME) is already present in the encoder - it's how P-frames get predicted from the previous frame. There are additional improvements to ME, such as quarter-pel searches and global motion compensation - these will be used in XviD sooner or later (the difference isn't *that* huge, but there is an improvement).

-h