Can someone help me encode without muddled color? [Archive]

View Full Version : Can someone help me encode without muddled color?

Cornflake

24th March 2009, 22:24

Hi everyone,

This is my first post on doom9 but I've known for a long time (doing numerous google searches) that you have some of the most knowledgeable people of video encoding / decoding so I decided to finally post :)

I'm trying to convert a bunch of recorded videos of high-resolution video game footage. I used divx for a long time, but I was never quite happy with the quality. So I decided to start encoding in AVC/H.264 because it should yield a quality increase -- and I've seen some amazingly beautiful stuff encoded by H.264 -- but I'm having some problems.

First, the quality. I have some videos of forested scenes with lots of detailed greenish colour (trees, grass, etc). But even when I encode my video at 10Mbps (The maximum allowed by FFDShow Video Encoder for some reason) I get lot's of artifacts. Not really mosquito noise or color range compression in solid colours like the sky (That's an improvement over DivX) but I get very noticeable blocking. VERY noticeable. And, one of the biggest annoyences I had with DivX was similar colours all look muddled together. It seems to lack the quality I expect from a high bitrate advanced codec. What am I doing wrong? I will give you a picture to illustrate (These are all 1920x1200):

Original Capture (Fraps video @ 298Mbps)
http://users.d2k5.com/thimg.php?f=Cornflake/files/obliv_original.png&w=150&h=150&r=0&flou=0&b=1&a=45&r0=255&g0=255&b0=255 (http://users.d2k5.com/Cornflake/files/obliv_original.png)

Muddled grass with blocking (Notice in the trees you can see solid squares of green, really obvious) H.264 @ 10Mbps
http://users.d2k5.com/thimg.php?f=Cornflake/files/obliv_blocka.png&w=150&h=150&r=0&flou=0&b=1&a=45&r0=255&g0=255&b0=255 (http://users.d2k5.com/Cornflake/files/obliv_blocka.png)

After staying still for a few seconds, blocking disappears and so does some of the color artifacts (Taken from same video as above)
http://users.d2k5.com/thimg.php?f=Cornflake/files/obliv_blockb.png&w=150&h=150&r=0&flou=0&b=1&a=45&r0=255&g0=255&b0=255 (http://users.d2k5.com/Cornflake/files/obliv_blockb.png)

And when you're watching the video in motion, it's way noticeable. Makes me want to gouge my eyes out. Would anyone mind telling me how to reduce the blocking and muddled colours? Maybe a quick walkthrough on how to setup deblocking and quantizer settings in FFMpeg video encoder? I mean, I know gaming footage is harder to compress and look good but this is certainly not what I expected. I expected sharp, clear, relative lack of obvious compression at 10Mbps. Thanks. :cool:

Snowknight26

24th March 2009, 22:28

10Mbps is definitely low.

Dark Shikari

24th March 2009, 22:28

You said you're encoding in H.264, but then you say you're using FFDshow's encoder. Which is it--or are you seriously using the (incredibly broken) x264 builtin in FFDshow?!

Try a real encoding GUI, like MeGUI or Ripbot264.

Inspector.Gadget

24th March 2009, 22:32

Try not using a VFW encoder, as using a modern CLI encoder will get you better troubleshooting advice. Look in the MPEG-4 Encoder GUIs forum for an overview of each. Sounds like your problem might be partially in colorspace conversion, though, which can be worked out in the AVIsynth forum if you still have that problem.

Cornflake

24th March 2009, 22:37

I'm using FFDShow Tryout's encoder. What I meant by H.264 is that I'm encoding in AVC. And I wasn't aware that it was 'broken'.

I've been hard pressed to find any good encoders. The one in Sony Vegas only allows you to save in .MP4 format, and It's not flexible. When using FFDshow's encoder I'm limited to saving as AVI because I don't have a program that would let me encode in MP4 and choose a 3rd party encoder.

I'll try those progs you suggested. The reason why I haven't found any good ones as of yet -- well, do a google search of h.264, x.264, or avc encoders and see if you can find anything in that mess! :p

10Mbps is definitely low.

Yes I planned to encode at 20Mbps but for some utterly unknown reason, FFDShow allows a max of 10.

Although I must add, I've seen some absolutely gorgeous results with 8Mbps so I don't know why mine looks so bad.

Dark Shikari

24th March 2009, 22:40

I'm using FFDShow Tryout's encoder. What I meant by H.264 is that I'm encoding in AVC. And I wasn't aware that it was 'broken'.

I've been hard pressed to find any good encoders. The one in Sony Vegas only allows you to save in .MP4 format, and It's not flexible. When using FFDshow's encoder I'm limited to saving as AVI because I don't have a program that would let me encode in MP4 and choose a 3rd party encoder.There is only one free encoder, and that's x264. All free encoding applications use it, and most cheap paid ones do too (with the exception of Recode, DivX, and Quicktime).

Any version of x264 that claims to be accessible through Video for Windows is inherently broken, as H.264 is inherently incompatible with Video for Windows. Use a GUI.

Esurnir

24th March 2009, 22:41

try with m-e-g-u-i. Really... If you need help setting it up just send me a pm.

Cornflake

24th March 2009, 22:45

Alright, I'll start setting that up and give it a try.

In the mean time, can anyone explain to me what exactly the 'Loop Filter' is. It's a setting you see in nearly every 264 decoder, and sometimes in encoders, too. I can't seem to find much info on it. All I know is disabling it is not ideal, produces a lot of playback artifacts (although greatly reduces CPU load).

- Thanks.

Esurnir

24th March 2009, 23:59

LoRd_MuldeR

25th March 2009, 00:21

I'm using FFDShow Tryout's encoder. What I meant by H.264 is that I'm encoding in AVC. And I wasn't aware that it was 'broken'.

If you were using VirtualDub with VFW ebcoders in the past, you may like Avidemux:
http://forum.doom9.org/showthread.php?t=126164

It's one of these "free tools" that use x264. Note that you need to use Avisyth input to open your FRAPS video. The same applies to MeGUI and most other tools.