how do I set quality to maximum?

[jsmith6]

I cannot find the proper settings for the Xvid encoder (version 1.1.0 on Windows, Koepi's build (http://www.xvidmovies.com/codec/)), in order to have maximum quality in my 1024x768px short movie project.

The project is in fact a slideshow with a number of effects (for example Gaussian Blur, Glow, Noise, Min and Max, Color Curves, and so on).

Althought I have found a combination of settings that preserves a good part of the quality, there are some areas of the picture (I mean the encoded Xvid file) that appear "regioned"... especially the blured ones.

The settings that I use are:

Profile @ Level: (unrestricted)
Profile
Quantization type: MPEG-Custom
Edit Matris: (I tried "hvs-best-picture.txt" and "ultimate-matrix" from the packaged matrices)
Adaptive Quantization: yes
Interlaced Encoding: no
Quarter Pixel: no
Global Motion Compression: no
B-VOPs: no
Max consecutive: 2
Quantizer ratio: 1.50
Quantizer offset: 1.00
Packed bitstream: yes
Level
Profile @ Level: (unrestriced)

Aspect Ratio
Pixel Aspect Ratio: Square (default)
Encoding type: Twopass - 1st pass
Stats filename: d:\video.pass
Full quality first pass: no
Discard first pass: yes
Zone options
Start frame: 0
Weight: 2.00
Quantizer: not selectable
Begin with keyframe: no
Cartoon Mode: no
Grayscale encoding: no
Chroma optimizer enabled: yes
BVOP sensitivity: 30
Quality preset: (user defined)
Motion
Motion search precision: 6 - Ultra Hight
VHQ mode: 4 - Wide Search
Use VHQ for bframes too: no
Use chroma motion: no
Turbo: no
Frame drop ratio: 0
Maximum I-frame interval: 300
Qunatization
Min I-frame quantizer: 2
Max I-frame quantizer: 31
Min P-frame quantizer: 2
Max P-frame quantizer: 31
Min B-frame quantizer: 2
Max B-frame quantizer: 31
Trellis quantization: yes
Other options
Encoder
FourCC used: XVID
Print debug info on each frame: no
Display encoding status: yes
Decoder
Brightness: about at middle
Post processing: everything besides "film effect" is enabled
Common
selected: Automatically ditect optimizations
Encoding type: Twopass - 2nd pass
I-frame boost (%): 10
I-frames closer than... (frames): 1
...are reduced by (%): 20
Overflow control strength: (%): 5
Max overflow improovement (%): 5
Max overflow degradation (%): 5
High bitrate scenes degradation (%): 0
Low bitrate scenes improvement (%): 0

I have created a very brief (3 seconds) Xvid file (http://briefcase.pathfinder.gr/download/545143) with a static picture to show the issue. The source file used to generate this Xvid is a PNG file (http://briefcase.pathfinder.gr/download/545142) (it's in fact a screenshot of the forum's splash page).

What do I need to change in order to get the maximum quality in encoded video?

[shon3i]

Profile unrestricted
Single Pass @ Q1
no bframes

[Sharktooth]

Q1 with h.263 quantization will only increase the filesize to incredible levels and wont give so much quality over Q2.
Set the Quantizer at 2 (or 3) and use a high-bitrate custom quantization matrix like 6of9, EQM V3UHR, EQM V3EHR...

[drcl]

what is the bitrate you are using?

If its just a slideshow, you dont really need to worry about the final size of the file (unless you are trying to get it on to a floppy disk!) so a contstant quality single pass is best. You dont need to use a custom matrix but it will help a little to use a high bitrate one like suggested.

The filesize shouldnt be huge anyway, i would assume alot of it is static screen grabs.

Weight of 2.00.... not sure what effect that has. leave it at default 1.00

[jsmith6]

what is the bitrate you are using?
Maximum, in all cases.

This is what I tried:

- used the ULTRA-QUALITY "eqm_v3uhr_rev2.xcm" matrix that I found in this post (http://forum.doom9.org/showpost.php?p=551686&postcount=4).
- set the Quantisizer to 1 (it may be an overkill but at least I am sure it's not this one)
- set bitrate to maximum
- set Weight to 1

I also tried single pass (setting quality to 1).

In both cases the "regioning" was still present in the blured areas of the picture.

I am wondering if this is the maximum quality that the codec can produce. I was hoping for something like the quality of pictures when converting them to PNG or JPEG (while setting every option to maximum). Are my expectations unrealistic?

[Sharktooth]

no... more quality (and a colossal filesize) can be obtained with EQM V3EHR.

[henryho_hk]

- set bitrate to maximum

Please turn off "Adaptive Quantization". And there is no need to set any bitrate in constant quantizer mode. You may also want to select the "Unrestricted" Profile to avoid bitrate caps.

As for the banding, would it be the old imfamous DirextX YV12 display issue? Would you please see if there is any problem in snaphots PNGs made by Virtualdub?

[jsmith6]

no... more quality (and a colossal filesize) can be obtained with EQM V3EHR.

Pardon me, I missed this one on that page.

As for the banding, would it be the old imfamous DirextX YV12 display issue? Would you please see if there is any problem in snaphots PNGs made by Virtualdub?

Yes, that was it. Thank you.

I am facing two more issues:

1. When viewing the resulting video, at some points, I can see vertical lines of one pixel height. I keep the "Interlaced Encoding" on Profile (unrestricted) turned off.

2. Red areas (for instance a red, sharp piece of text on pitch black background) seem a little blurry.

What can I try for those?

[henryho_hk]

1. Interlaced encoding is only for interlaced contents (e.g. movies, TV captures)

2. Dunno how severe the blurriness is...... possibly a limitation of the YV12 colorspace (the chroma resolution being 4 times lower than luma).

[jsmith6]

1. Interlaced encoding is only for interlaced contents (e.g. movies, TV captures)

I keep it disabled, yet at certain points where image effects come in (Glow, Noise, etc uppon static images) the resulting video has these vertical lines. What I want is to prevent those vertical lines from appearing.

2. Dunno how severe the blurriness is...... possibly a limitation of the YV12 colorspace (the chroma resolution being 4 times lower than luma).

Here is a sample: http://img421.imageshack.us/img421/8303/test2hf1.th.png (http://img421.imageshack.us/my.php?image=test2hf1.png)

The text on the top is from the Xvid video. The text on the bottom is from the source.

Can I use something else besides YV12? I haven't seen any option about that.

[foxyshadis]

What is the source? RGB? YUY2/YV16? In either case it will be necessarily transformed into YV12 for encoding; while some MPEG-4 codecs allow you to encode to YV16 or YV24 (planar 4:2:2 and 4:4:4) very few players support it. This kind of situation is too rare for the effort and increased processing time and bitrate needed for most to support it.

The transforms will then have their way with it, in which case besides using a matrix for it, you'll probably need a q1 or q2 zone wherever the text is.

[Prettz]

I keep it disabled, yet at certain points where image effects come in (Glow, Noise, etc uppon static images) the resulting video has these vertical lines. What I want is to prevent those vertical lines from appearing.



Here is a sample: http://img421.imageshack.us/img421/8303/test2hf1.th.png (http://img421.imageshack.us/my.php?image=test2hf1.png)

The text on the top is from the Xvid video. The text on the bottom is from the source.

Can I use something else besides YV12? I haven't seen any option about that.

If your quality is already pretty much maxed out and you're still getting that on the text, maybe you could try upsizing your resolution with Lanczos4Resize? MPEG codecs are fairly brutal when all of your picture information comes from the chroma channels, and the only way to effectively fight it is to just give the codec more picture information by increasing the resolution. Since you said filesize was not a concern, that could be an option. You could probably do away with using Q1 and just stick to Q2 if you raise the resolution high enough.

[Blue_MiSfit]

yeah YV12 kinda sucks :D

It's perfectly fine for most things, but if only everything was 4:2:2... not that I'd like to pay for the extra time/bitrate to deal with it, but still :)

The 'bouncing red ball' scenario just upsets me to no end.

~MiSfit

[Prettz]

It's perfectly fine for most things, but if only everything was 4:2:2... not that I'd like to pay for the extra time/bitrate to deal with it, but still :)

What would be even cooler is if a codec let you just switch between 4:2:0 and 4:2:2 using zones. Just require all decoders to have enough surface memory at hand for 4:2:2 all the time. Then an HVS system in the encoder that automatically switches between subsampled and non-subsampled as needed to avoid ugliness in dark scenes.

[jsmith6]

What is the source? RGB? YUY2/YV16? In either case it will be necessarily transformed into YV12 for encoding;

I am not familiar with the terms "YUY2", "YV12/16/24". I assume that they are all something like RGB, a way to present the color.

The source files are still images (slides) at RGB. I am not sure what the software that does the rendering exports but I have compared a rendered still in a PNG format and looks exactly the same quality as the source file. So does this mean that it renders in RGB too?

while some MPEG-4 codecs allow you to encode to YV16 or YV24 (planar 4:2:2 and 4:4:4) very few players support it. This kind of situation is too rare for the effort and increased processing time and bitrate needed for most to support it.

You mean hardware players? I am not currently interested in them.

This is going to be a very specific situation. I am not that interested in keeping the output video file in compliance with MPEG-4. I intent to play this file on a PC with the Xvid coded installed. It will probabably be a Linux liveCD (Knoppix for instance, Windows is out of the question since since the image makes a bending).

What would interest me is to require the least of system resources, in order to enable old computers to play the file. But even this is secontary.

The transforms will then have their way with it, in which case besides using a matrix for it, you'll probably need a q1 or q2 zone wherever the text is.

I already keep Quantisizer to 1.

If your quality is already pretty much maxed out and you're still getting that on the text, maybe you could try upsizing your resolution with Lanczos4Resize? MPEG codecs are fairly brutal when all of your picture information comes from the chroma channels, and the only way to effectively fight it is to just give the codec more picture information by increasing the resolution. Since you said filesize was not a concern, that could be an option. You could probably do away with using Q1 and just stick to Q2 if you raise the resolution high enough.

If I understood correctly, AviSynth is a framebuffer. So I am going to "feed" the rendering to AviSynth which will then resize (enlarge) it and then encode it with Xvid.

Yes, that is an option, although I am concerned about what kind of a system it will need to playback this video. And it would be preferable to fit the project in one 4.7GB DVD. But it is definetely an option. I will weight the benefit and the cost.

So far, I have been rendering into 1:1 size, meaning my sources are 1024x768 or 1280x1024 and the output video is 1024x768. What would seem to you a sufficient enlargement? 20% 40%?

[foxyshadis]

What YV12 means is that all color is averaged in each 2x2 block. (YUY2/YV16 means each 1x2, averaging horizonatally.) Most graphics engines will point-resize it back up, which sucks horribly. Some ATI and Nvidia driver versions will smooth it a bit more on playback though.

I just took your sample pic and loaded it into avisynth; by converting it to yv12 it causes exactly the same effect your "after" half shows. That bites. Resizing will introduce its own artifacts and problems, so I doubt that'll work terribly well, though you can always try a 50% enlargement and see if the effect is reduced enough. (Note, however, that if your video is too large for the screen and has to be sized down, some systems will resize it down in yv12, eliminating the benefit.)

My advice: If you can't layer them on top during playback (softsubs, like SSA/ASS, are entirely capable of this and can work in RGB in MPC or vsfilter), you should simply encode losslessly to huffyuv or ffv1, both of which can handle RGB.

[henryho_hk]

If you want to stay in MPEG4 ASP, you can also try luma & chroma sharpening during playback. The only question is how to do it in GeeXboX (Live Linux for multimedia).

MJPEG codec seem to support RGB and YUY2. But then very probably they are not free, and they are not MPEG4.

[jsmith6]

Unfortunatelly it is impossible to layer the text, since it is in fact different layers of text with effects on it during the whole timeline.

I tried huffyuv. It produced a realy colosal filesize but I have to admit that the quality was crystal :) I think I will just settle for Xvid. I hope that they will add RGB in the future.

As for the banding, would it be the old imfamous DirextX YV12 display issue? Would you please see if there is any problem in snaphots PNGs made by Virtualdub?

I tried playback in Linux (Ubuntu 6.06 and Knoppix 5) and the banding was present (although red and colors that approach red didn't distort like they do in Windows). Additionally, playback under Linux is much more slow (especially on scenes with a lot of effects) and interlaced(*) while the same scenes are perfectly smooth (and of course not interlaced) under Windows. I have the Nvidia drivers installed on both operating systems (tested, I can play UT2004).

VirtualDub keeps displaying still frames without bending.

What went wrong?

(*) I didn't render in interlaced mode, it only appears in Linux.

[foxyshadis]

In Linux, it's probably getting a higher quality scaling in software, via mplayer or gstreamer, but that doesn't use the video card at all.

Unfortunately Xvid can't ever support RGB, actually, because the MPEG specs don't allow it, and probably won't ever support 4:4:4 YUV (the next best thing) because decoder compatibilty would be almost none. x264 probably will, however, since akupenguin is interested.

[Blue_MiSfit]

That would be insanely cool and useful for a production environment!