Fog is blocky

[Ryuudo]

http://personal.inet.fi/clan/ryuudo/fog.jpg

How could I make the fog look more realistic and less blocky?
I've tried koepi's newest dev. build and the stable build with several different settings, but this is the best I have gotten so far :(

[Teegedeck]

The link you give is dead (ATM at least).

quoting from here (http://forum.doom9.org/showthread.php?s=&threadid=48124):

You're supposed to let people know your exact settings if you really expect comments. Please read the 'posting bugs'-sticky.

[Ryuudo]

Originally posted by Teegedeck
The link you give is dead (ATM at least).

quoting from here (http://forum.doom9.org/showthread.php?s=&threadid=48124):

Lol, I forgot to upload the image :D. It should work now.
Sorry 'bout that topic "posting bug" -sticky...

[bond]

I think Teegedeck meant that you should post your 'exact settings if you really expect comments'

[Didée]

Without knowing what you have done:

Try mpeg quant, not h263.

[Ryuudo]

Ok, sorry. I wrote that message in a hurry so...
Anyway...
I'm using H263 as Didée guessed.
I don't use alternate curve and other settings are default except I use 16 Max I-Frames, 31 Max P-Frames, No qpel, gmc or b-frames other unstable features.
I use the newest Koepi's dev build (XviD-17022003-1).

[Teegedeck]

What kind of (if any) filtering did you use?

What bitrate or filesize? What quantizers does running debugview show in 2nd pass?

[Ryuudo]

Hmm... let's see...
Smart Resize (DVD source so...)
2dCleaner and Warp Sharp
File size is 193mb.
I'll check quantizers tomorrow...

[Ryuudo]

What quantizers does running debugview show in 2nd pass? [/B]

I encoded it again with mpeg, but it's still blocky. Debugview showed quantizers 2-6 most of the time.

[Teegedeck]

Throw all these denoisers out of your script and try again. May help.

[Ryuudo]

Originally posted by Teegedeck
Throw all these denoisers out of your script and try again. May help.

Ok, I'll try w/o filters and use mpeg quant. I'll post results when the encoding is finished. :p

[crOOk]

I just encoded a movie (quant2) that started with a scene with lots of steam. Turned out horrible. The objects surrounded by the steam were extremely blurred Removed all the filters. Still awful. Removed all the fancy stuff, even the b-frames. No difference. Took a look at the mpeg2 source, looked brilliant (PALDVD: "Auf Herz und Nieren"). Tried different matrices... Conclusion: Won't encode films with lots of steam/fog nomore... ;)

[Didée]

@ crOOk

Since you seem to have a particular clip where steam/fog turns out so horrible:
If you're still in mood for testing, could you give a shot with different, speak lower, motion estimation?
I often think that XviD's ME is too active on some content, e.g. smoke, seeing motion where is none.

[DBaT]

Thanks Didée for the idea for trying lower ME.
I have a similiar problem with a scene where two fellows sit in a small train toilet talking and from the window comes a flickerin light. The whole scene goes totally blocky with H263@1100kbs.
I have tried everything I can figure but nothing has helped.
It can ofcourse be just a hard to encode scene as with quant2 I dont have the problem but I have never run to anything like it even with scenes with water or fog/smoke. Have to give a new try with the new build.

[Koepi]

Flicker is something different then fog - in flickering you get blocks because the massive use of intra(=key-)frames. In foggy scenes there are no intraframes eating up the bitrate.

Regards
Koepi

[Ryuudo]

[4080] Q:2:427
[4080] Q:3:13539
[4080] Q:4:15707
[4080] Q:5:3580
[4080] Q:6:143
MPEG, Curve Compression High: 25 Low: 10
No filters...

[DBaT]

Thanks Koepi that makes perfectly sens what you say.
Something new learnt again, time to figure out something else.

[azra28]

All mpeg4 codec have same pb with fog or water, why? I couldn't answer, but I always saw the same things

[Ryuudo]

Well, I'm encoding it with new settings now. I'll post screenshots from the fog part tomorrow.

[Sirber]

Hi

could you try with RV9 too? I don't know how the fog looks like with this codec. I have no movie with fog to test :(

[Shinobu]

The fog is realy hard to encode , on movies with lot of fog I usualy use a lower resolution (512*XXX) with bicubicresize (0.5) , no filter just cpu=4 in meg2dec3.dll (marcfd version) and trie to encode with 1-pass quantitiser and quatitiser 2 + h263 + MSP=6 + VHQ=1 + CM + QPel + max bf=3 + Bf %=150 + Bf offset=100 + bf threshold=50

With a little luck on short movies (<1h45) you'll get a fille <630 Mo (just give 70 mo for sound)


that's the only way to me to get a "perfect loocking fog" on a movie

Try it .... with a little luck...


++

[willie1]

My experience with fog has been to increase bit/pixel ratio by reducing framesize or increasing bitrate. Leave lumi masking off. It will reduce the bitrate in these areas as they are recognized as lighter. Use a resize filter that recognizes detail better and does not blur it. Lanczos or sharp bicubic work well. Mpeg quantizers help over h263. Use a normal curve compression as the changes in the fog may be seen as motion. A bit/pixel ratio of 0.25 has been my lowest that I found tolerable. Hope this helps.

[yaz]

hi ryuudo (& all)

yes, fog (steam, dust, ...) is one o the hardest stuff for mpeg4. it's mainly cus of the low luma differences (lumanoise) within the macroblocks. another point is the decision what is motion & what is luma difference (texture) within the scenes like this. shortly, the codecs usually find these frames xtremely compressible, in a meaning of removing more details than in other scenes. the solution, in general, is decreasing compressibility. but ...
if it's just a short part u'd better try to live with it. there's no use of wasting weeks for optimizing some hundreds frames. (i'd do it but most wouldn't :-)
if it's a longer part it's worth to pay attention. some tricks working well for me l
trim that part & apply a different filter/codec setting for
- lower (or leave) noise removal. forget heavy guns like con3d but, imho, 2dclean is also too much here
- use extra sharpen but some kinda 'neutral' one. i mean core sharpen or unfilter but forget warp- or msharp. asharp may also help in a way
- my favourite is blockbuster/sharpen. it improves contrasts only in the low lumanoise regions giving a more realistic look to that scenes on encoding.
- use mpeg quant not h263

some really dirty tricks (when any other seems hopeless :-)
- apply framecount proportional target size for that parts. this way the codec is forced to use more bits on these parts than it would do on encoding the whole movie at once.
- restrict quantizers heavily
- try q1 too
- ...
these all try to pump more bits into these scenes than dictated by the bit distribution algo. but it will work only if the codec feels that there is really something to encode! increasing bitrate wont solve the problem alone. so apply these tricks in con with (some of the) aboves.

btw, the pic u showed is not the worst i've seen. if u want a tough playground try the jabba scenes from return of the jedi. i got a tv capt of that. it's noisy, blocky, whatever. made me headache of weeks to figure out how to make it.

the bests
y

[Ryuudo]

http://personal.inet.fi/clan/ryuudo/fog2.jpg
http://personal.inet.fi/clan/ryuudo/fog3.jpg

I will try those that yaz and shinobu suggested this weekend. I'll keep you guys informed :p

[Sirber]

Looks like someone is sick :) Greeny fog...

[Ryuudo]

fog pictures from the dvd (http://personal.inet.fi/clan/ryuudo/fog.rar)

I packed some images from the dvd to that archive for comparison.

[crOOk]

Originally posted by Didée
Since you seem to have a particular clip where steam/fog turns out so horrible:
If you're still in mood for testing, could you give a shot with different, speak lower, motion estimation?
@ Didée
Thanx for the advice. Unfortunately I've already deleted the VOB files from my harddisk, but as soon as I get ahold of the source again, I'll try your suggestion regarding a lower motion estimation. If anyone else is interested in the source that has such great amounts of steam over a quiet long amount of time in the first scene, it's a german PAL DVD and the movie's name is "Auf Herz und Nieren".