Film grain harder to encode?

[Mango Madness]

Most of the time i test new builds with the almighty Attack of the Clones and am able to produce quality movies at 800meg with all the extra goodies enabled with a resolution of 720x306 (after cropping and all that). But now i've whipped out my Band of Brothers DVDs and noticed that they are particularly hard to encode. I've noticed that the film grain is pretty bad and wonder if this has an adverse effect on compression. I've tried to make a 650meg file (1 hour 12 minutes) at a resolution of 712x400 with all the extra xvid features (bframes 3/150/100, chroma, lumi, gmc, vhq mode 1 and 4 for 2 different tests, h.263 matrix) and the picture is noticeably worse (smearing and motion of macroblocks) even when i've applyed convultion3d to the movie.

[iago]

Originally posted by Mango Madness
I've noticed that the film grain is pretty bad and wonder if this has an adverse effect on compression. Of course it has! ;)

[willie1]

I encoded this series as well and it was difficult to compress. Do not use lumi masking for this type of source. It will create macroblocking in all the hazy or foggy scenes. Divx's PVE has the same effect in fog, haze or rain as well. Changing the res to get a bit/pix ratio of 0.25 was necessary to have a block free series. Anything below 0.25 resulted in blocks unless I used excessive processing which reduced clarity more than reducing the res did. I did keep the ac3 audio so my res was a little lower than normal for some of the episodes considering the shorter length of the clips compared to a 2h movie. Episode 2 Day of Days was the most difficult and required a res of 512x272. Light noise reduction in avisynth helped.

out of curiosity, what video card do you have? I have a nvidia geforce 3 and my Band of Brothers looked great whereas my brother in law has a radaeon 8500. His card producesd macroblocks throughout the series where mine was clear. His card has greater difficulty with "dirtier" sources. My brother's geforce2 mx400 is clearer for these scenes as well. I'm not knocking ati products but there was a distinct difference with this type of clip.

[TheXung]

nVidia cards will show more macroblocking if you turn "Digital Vibrance" up. It's also a function of the contrast settings on the display.

[Mango Madness]

I have a Rage Fury 128 baby! But i'm a senior and i'll be building a new system with all my graduation money so i'll hopefully get a super nice video card. I have a 22" monitor so it's really easy to see all the muck. I'm testing a more conservative settings arrangement at the moment:

Umaniac Feb 21 build
Avisynth (convolution 3d set at Low Quality and Fluxsmooth)
Resolution 712x400 (aiming for about a 650meg video file muxed with mp3 vbr preset medium LAME 3.94 alpha 11 build)
Virtualdub 1.5.1

2 pass internal
Motion search 6
H.263 matrix
Xvid FourCC
VHQ 1
Chroma Motion
(no bframes, gmc, or lumi)

[willie1]

I tried higher resolutions with this movie with no success. This is a case where less really is more. i also found using sharp bicubic to resize and keeping the bit/pix ratio higher rather than using bilinear or soft bicubic with a higher res (normally compresses better) workes better for this series. The "greyness" in the night scenes, fog, and rain were preserved better in this fashion. The softer resizes left a banding effect in the grey areas. i feel the biggest detriments were lumimasking and too low bit/pix ratio. I didn't have problems with GMC or Q-pel though so use them if you want to take advantage of the increased bit savings.

Even though i only have a few posts here, I have over 850 at Divx.com under the name warrenkrywko. I am new to xvid but not to mpeg4.

[Didée]

Originally posted by Mango Madness
[...] 800meg with [...] a resolution of 720x306
[...] 650meg [...] at a resolution of 712x400
Mango, you are very optimistic when choosing your resolutions!
I see you're only resizing vertical, but although this may improve "relative compressability", we still have the good old "bits/pixel*frame" thing ...
E.g. for Attack of the Clones, I had to drop to 608*256 until I reached the quality level I'm aiming for!
Give the "Clones" another run at 640*272, and compare yourself!

[droolian01]

Hi all

I captured this series with my dvb-t card and found it very difficult to compress too. I ended up fitting one ep per 700mb cd at 592x336 using Lancross resize using vanilla xvid (no b-frames etc...)

But the most important thing to remember is that the makers paid a lot of money to make it look grainy and to give it a washed out look. They wanted to recreate a newsreel footage feel.

So, my advice is to reduce the res, DON'T SMOOTH (possibly a light conv3d at most) and consider b-frames if things get too blocky (say 2,15,100). Doing anythin else would be like artificially recolouring Shindler's list!

droolian

[iago]

Also, remember that b-frames with even light settings might produce very noticable blockiness in "high-motion scenes", despite helping compressibility a lot and thus improving the general look of the encode.

[sysKin]

Originally posted by iago
Also, remember that b-frames with even light settings might produce very noticable blockiness in "high-motion scenes", despite helping compressibility a lot and thus improving the general look of the encode. Which is why we don't put b-frames to high motion scenes, right? Correct me if I'm wrong, I actually have a feeling that our p/b decision is completely crap and we should put less b-frames...

Radek

[Mango Madness]

My attack of the clones is only for testing features of new builds so I'm not so worried about producing "release-quality" encodes but i can produce pretty amazing results even at those resolutions with the newest builds just cause the digital source material is very easy to deal with.

Also, I just finished with the test I described above with pretty conservative settings and it produced very nice results. The only problems I see is that the filters I used probably were a little too strong and the image just isn't as sharp but this sacrifice is well worth it. There is just a tiny bit of ringing around the characters but this is only noticable if you get right up to the monitor. I'm gonna perform another test with the aforementioned settings but with VHQ set at 4. Thanks for the feedback and I hope some of you try the settings i've mentioned.

[sam_b]

syskin, I do think that xvid could do with a bit more b-frame culling, but I don't think it's too bad. What about a dynamic multiplier, so that static frames use higher quantisation on b-frames and fast-motion frames use a similar quantisation to the p-frames? Is this a practical idea?

The other thing I was wondering is if it is possible to make the codec use a larger p-frame interval, as it does not seem to want over 3 b-frames put in. For DVB captured cartoons often 95% of the frame will stay *completely* static for several seconds at a time, and I don't see why the whole scene can't be done with b-frames, leaving low quants for pans and I-frames. Any thoughts?

[sh0dan]

@sam_b: For static scenery there is no point in using either B or P-frames - both compress the differences very well. (Remember P-frames are also based on the previous images, so they also have the ability to simply leave an unchanged macroblock alone)

B-frames are good when a scene is quite static and it at the same time is able to predict what happends between two P-frames. As the P-frames becomes more different, the more will have to be coded into the B-frames (at lower quality).
Using a few (one or two) B-frames between P-frames also have the advantage that the lower quantizers will only be visible a short time.

But there is a "trade-off-point" with B-frames - and being able to make XviD more conservative on inserting B-frames could be a very nice addition - but I guess that testing how good the motion estimates are between two P-frames (compare the "predicted" B-frames vs. the correct ones) are quite expensive.

[Bulletproof]

I've had a good expierence with b-frames, although they seem to add some bad blocks to the video with the current builds (known already), they still offer a good filesize decrease. Personally I feel B-frames don't improve quality, rather, they slightly decrease quality (depending on your settings) but offer a good filesize benifit.

In my personal opinion, the old I-P frame encoder with no extra options seems to look better than w/ Q-Pel, B-frames, Luma, GMC, VHQ. However, the filesize will be bigger, and difference in visual quality isn't THAT large when enabling the new options, so it still looks pretty great, but the filesize difference is.

For the question on Film Grain, don't use the H.263 matrix because this will throw away the grain information and turn them into blocks. Try using the HVS matrix.

[TheXung]

Perhaps b-frame decisions shouldn't just be based on amount of motion. Perhaps there should also be a noise factor taken into account. Maybe a threshold SAD or something

[sam_b]

Thankyou for your reply sh0dan. The reason that I wanted B-frames specifically is that it the ringing which is inevitable on a cartoon source at a low bitrate is non-correlated between frames, as a B-frame has no reference to the previous one. To my eye this look significantly better as the mosquito noise produced is 'fresh' each frame. Does this have any basis in reality?

In any case, the bitrate can still be much lower with similar quality than I can acheive with P alone.