The Blue Aurora Borealis That Would Not Die

[sadie]

Five day wait to post here. Whew. Well you brilliant people are my last resort. OK, after dozens of tests with Xvid and Dvix, virtualdubmod, filters up the ying yang the above feature remains a permanent element in the daytime landscapes of my videos. So at the risk I being redirected to that one esoteric sticky I probably missed, here goes. I post my query in two parts, Part A for the impatient that I believe gets straight to the point. Part B for the background profile of this 'case'.

A:
Is there a filter that allows one to select and smooth the color Blue. Meaning: To appoximate what Photoshop and Paintshop do with their 'magic wands' selectors. And Abis: Is it possible to use a filter that will only be applied in zones defined in the Xvid?

B:
Material: Panasonic GS400 (3CCD), Sempron 1500, 750 mb ram, capture over firewire, Pinnacle Studio 9.4

Filming Characteristics: Handheld (w/ or w/o phony stabilizer), penchant for long reverse zooms, pans, short 3-5 sec cuts, overuse of dissolves, interspersed stills, multiple sound tracks, very sparse use of color or speed correction) Use wide-angle adapter most of the time, sometimes with polarizing filter. Voila!

Object: Reduce DV AVi edited and rendered in Studio to Xvid compacted to 1/3 mpeg2 size(1.5 gb) at 720x576. De-interlaced or not.

Problem: waving swath of small blue blocks in primarily solid colored areas that seem to manifest themselves at color gradation points.. Details scenes, no prob.

Flashback: Original DV AVI seems to exhibit fine noise (3CCD?). mpeg2 made from AVI has blue blocking although much less so. I.E. problem may lie in source.

Only Solution to Date: Encode in very high bit rates (4000mbs+) nearly eliminates problem. In fact best solution was to make another copy of the DVD. Translation: solution defeats purpose of mpeg4.

Trials and Error:

- De-interlacing techniques (smart, area+smart, kerneldeint) do nothing. Smoothdeinterlace resizes image to 360x512 ?!?! No thanks. Prefer not to de-interlace but price at the compressibility counter is too great to pay.
- Denoisers do even less since they don't deinterlace.
- Smartsmoother doubles or triples encoding speed and partly alleviates the problem while completely alleviating the quality of the rest of the image. The classic problem.
- Two Pass vs One pass has no effect. I use single pass anyway and since precise file size is no concern. Anyway, I've yet to see a difference between the two methods.
- Varying Quant factors, sometimes gets xvid sped up enough to attack problem, but since problematic scenes are relatively static it is of no practical help. Anecdote: trying to transition a blacked-out music-backed intro of 15 sec into a sucession of blue sky scenes it took nearly half a minute for xvid to catch up to even 1500-1700 mbs at Q3 or Q4. And at Q5 we're in triple digits. Xvid is no drag racer.
- Using Zones does help, but defining 30-40 blue sky zones of 5-20 seconds is hardly an elegant solution. And since Xvid (AFAIK) does not allow you fix your bitrate within zones, 5 seconds is hardly enough to wait for xvid to go through its gear-shift routine. I admit I've never used the 'Weight' option.
-attempts to resize to 640x512 (I'm in PAL-land) improve things, but only because I can use highter bitrates and not sacrifice file size. But although it's a close call, the native resolution is better enough that I prefer not to modify it.

Odds and Ends: Colour space doesn't seem to come into play (ffdshow shows yuv2 universally). Or does it? I understand the basic theory behind this topic, but have yet to see any real life, real time manifestation of it. Which proves I don't know much about it after all. Playback varies within players. My default zoomplayer (ffdshow filter) exhibits phenomenon most and gives slightly washed out color. If I attempt to switch to the Xvid decoder ZPlayer spits open not one but two active X windows one of which plays the video and neither of which can be close without resorting to the task manager. BSPlayer & WMP9 seem to display much more dense color (brightness, saturation?) at default settings and the blockiness almost disappears (discovered this yesterday). Why? On my standalone DVD/Divx player the blockiness is present but 'masked' behind the TV's improved color and contrast (It's a Trinitron, of course).

Final Detail: I omitted to clearly state that the problem does not appear in all blue sky monochromatic situations. And it doesn't either matter if there is movement involved or not. Haphazard would be the word. Also haven't analyzed differences in light quality, 2-hours b/4 sunset vs high noon, smoggy pastel blues vs monument valley pure azure. If so, things start to get out of hand.

Finished, and what a mouthful. Please any insight into this dilemma would be oh so appreciated. Thank you in advance.

[henryho_hk]

Please host some screen captures, or better, a 10s sample source DV clip and "bad" xvid clip somewhere.

[sadie]

After three days absence, I just read your post henryho. Ok I'm new to this 'hosting' business, but managed to send a 30sec clip (too few I-frames to cut it shorter) off to http://video.google.com. Search for the clip Romeintro 1. The web view is inadequate to pick up details, but you can download the file if you also download their little viewer. If you have any better suggestions please reply. This particular version of the clip is the best quality I've yet managed. If you want something where the anomalies are more flagrant just holler.

[communist]

Plenty of hosting solutions can be found here:
http://forum.doom9.org/showthread.php?t=96362

A sample clip that can be used to reproduce it would be very helpful. Because right now I'm not really understanding whether you're going DV->MPEG2->XviD or directly from DV to XviD.

[sadie]

Alright, let's try this one
http://rapidshare.de/files/36183661/Romeintro_1.avi

The file by the way is direct from DV. Mpeg2 reference was only made for quality comparison purposes.

[Mug Funky]

you could take a stab at turning off "chroma motion", and turning on "chroma optimiser".

sounds like your camera has noisy blue... most do (though 1-chippers have it much worse, it's probably overwhelmed by the other noise).

you could also try avisynth with a mild temporal filter on the chroma, ie:

temporalsoften(2,0,5)

that'll leave the luma alone, but give the chroma a fair bit of smoothing. shouldn't change the look of the video at all

unfortunately xvid has a tendency to make noise swim about quite noticably. mathematically it "looks fine", but flat areas with mild noise can really jump out sometimes.

[edit]

disregard the above... (for now). i just saw your sample on a TV and computer screen simultaneously... there was no problem at all on the TV, meaning it's just the conversion to RGB at playback that's letting you down.

try ffdshow with gradfun enabled and you'll probably get a nicer picture.

btw, nice camera! i've seen much worse than that on commercial DVDs.

[sadie]

Thanks for reply Mug Funky. I managed to get a hold of the gradfun filter. To be honest it does seem to attenuate much of the static banding. It does not touch upon the related problem of the upward moving 'curtain' that appears between the third and fourth scenes. Yes, the bands are less blocky, but the wave effect is undiminished. Turning up the volume in gradfun to 15 or more does nothing but add some genial halo effects around the statues. This rising curtain happens elsewhere in my video and will at turns exhibit itself as rising or falling depending on if the scene is dissolving in or out. The 'curtain' isn't apparent between straight cuts, but only where there is a transition effect employed. I wonder if there isn't a way to pre-process with gradfun in virtualdub. It would allow me to cut out the smoothing step.

From your comments, I gather the issue stems from a blend of the camera's deficiency (noise), xvid limitation (flat colors) and color space conversions. I'd been led to believe that 3CCDs were, on the contrary, more susceptible to noise than the single variety since their three channels are processed with separate mirrors and thus lead to more light dropout. Secondly, if xvid is quirky with solid colors, why do I notice the same problem withe Divx6? Or is this a general mpg4 weakness?

Color space. Ok, I'd love to be pointed to a video-related primer on this subject. You were quite right, the clip is clean when viewed on the TV (Yuv12?). But my initial unfiltered encodes did show the anomaly albeit to a lesser extent. So the issue then is one of degrees.

In fact, what is the output space of my xvid? I ask this because in FFdshow under BS Player I'm told it's YUV12. In Zoomplayer it appears as YUY2 (same as YUV2?). Moreover, at the risk of sounding like a dummy, why shouldn't the output simply show RGB which is what I'm viewing on the monitor, right? And while I'm on the subject, one of the reasons Avisynth filtering is supposed to be superior to Virtualdubmod is that it doesn't do a turnaround conversion YUV12>RGB>YUV12. As I prefer the ease of Virtualdubmod, is the difference in quality so noticeable that it really warrants the use of the more fastidious Avisynth?

You're being generous re/ my GS400. At times, I consider it little more than a fat silver cigar with built-in 'shake, rattle and roll'. Now if somehow only knew of a cheap steady-cam solution... Thanks for any further suggestions. I'm not out of the woods yet.

[henryho_hk]

Try:

tdeint(order=0)
degrainmedian(mode=2) #or if too strong, try mode=3

As Mug Funky said, you can try a low luma degrain and a high chroma degrain. If you host a small clip of the original DV avi, we can try it also.

[chilledoutuk]

i agree if we could have some of the source then we could really help you out.

By the way have you tried h.264 codecs?

[Didée]

Secondly, if xvid is quirky with solid colors, why do I notice the same problem withe Divx6? Or is this a general mpg4 weakness?

The latter, it is.

http://img141.imageshack.us/img141/8057/dilemmasd9.png (http://imageshack.us)

In the given case, visually you prefer (b). For the encoder, (a) and (b) mathematically seem equally good. If then (a) turns out to be cheaper to code, you get this "moving curtain" effect.
Encoders do not have human's brain to interpret the result.

Another typical case is walls etc. that are lit by flaring fire, where the illuminated textures start to move, instead of staying in-place and just change their brightness.

[sadie]

Ok, guys I'm back. Sorry if I'm not always on top of this issue. I've got this other enterprise called 'life' to deal with too. Thanks for the latest posts. Henry I'll give your suggestions a shot, although...Didee your little graph was cute, but out of context it really meant zilch to me. Proof again that this board is angled towards the afficionado. Struggling with the dynamic you've tried to illustrate I could only come up with a related metaphor. It made me think of how autofocus cameras have a next to impossible time settling on a focus point when the field lacks precise detail: fog, glass, water, etc forces the mechanism to search like crazy until at times it 'grabs' onto a point that may or may not correspond to the focal plane.

So you wanted a source clip. Alright, check the following link. http://rapidshare.de/files/36549219/Romesource_1.avi.html.
This one has the moving curtain effect going in different directions at either end of the dissolves. Of course you won't see it yet, since it is the source so you'll have to compress it yourselves.

I'm a bit discouraged as last night I encoded the entire film (about 1 hour) and discovered a half dozen other areas that need substantial tweaking, the sound that is clipping badly and my four players none of which agree on which image is the right one.

I suppose I'm going to have to buckle down and read a manual about video theory (still no feedback about the color space issue) Many of the links found on this site are dead.

But after having plunged into this maddening medium of mpeg4 for the past week or so, I'm nearly convinced that for home video (with all its inherent limitations), the xvid/divx diet just isn't worth it to shave off those few pounds at the end. Xvid/divx can probably do wonders if it is spoon-fed a 'princess-like' source (commercial DVDs). But for the quality I'm looking for, at 4000mb/s I think I'd prefer the ease of mpeg2. So the disc costs an extra eurodollar.

At any rate, even if I'm a neophyte, the theoretical and technical questions still fascinate me, and I would still love to hear if anyone can solve the mystery of the moving curtains.

Does anyone want to buy a used cat. It's at that age now where you're tempted to use a phaser on it.

[foxyshadis]

MPEG4 can still deliver the same quality as MPEG2 in one half to one third of the space. That's enormous compared to typical 800-1400 kbps rips you're probably currently aiming at, but it can still save you a lot of space, if you carefully follow Teeg's simple presets (http://forum.doom9.org/showthread.php?t=107897), and note that first link and the batch file at the end of the thread.

If you're willing to trade lots more space for quality, you won't have any of these shimmering problems, although you might want to get avisynth or another video processor involved to work on it. HDRAGC and perhaps the film look thread are especially useful with DV.

[sadie]

Hi, Foxy, I missed Teeg's link somewhere. With a bit of skeptical reserve, it looks like what I might need. I'll look deeper into it tonight. But to clarify, I can get half mpeg2-sized files that comes darn close in quality at 4000kb/s (pardon I'd said mb/s last post). But under that, the artifacts come out real quickly. At 1/3, circa 2600kb/s (the encode I just finished), static sequences like the one's I've posted are in a pinch decent enough. But others like a succession of rapid jump cuts weaving through a Mongkok street scene with passers-by coming and going or a static shot of a sea of fine-fronded palms jostling about in the wind, the whole thing just falls apart in a glory of fuzz and flutter. Thing is I know these are ultra difficult things to correct and I don't expect miracles. It's only that from my perspective of the trade-offs involved, half DVD quality is not good enough to balance the work the Xvid requires not to mention the compatibility factor with stand-alone players (although that's changing now). Remember, I'm doing this more as a techno-intellectual exercise than to be able to post stuff on the web.

But I'll look at Teeg's suggestions and if I can get my file down to 1/3, I'll eat my words and possibly even give the cat a new lease on life.

[foxyshadis]

If you're having issues with a bunch of palm trees moving in the wind, I can tell you from experience that CBR 9800 DVD-Spec Mpeg2 probably wouldn't be able to cut it from a total quality perspective either. If you have such a scene handy, I can give it to CCE, Tmpeg, HC, and Qenc just to see where they top out.

[sadie]

If you're willing foxy here's the link:

http://rapidshare.de/files/36641649/Australtest_1.avi.html

It's short and not very sweet but source heavy. So you won't see the smears. And of course taken out of the linear context, I wonder if any isolated treatment might not distort the results a bit. But I'd love to hear your findings.

A closer look at Teeg's thread had me asking what a Megui was. Ah, another new program to download. Just when I was settling in to Vdubmod. Whew. Dumb question but can h264 be 'read' in stand-alone anywhere outside of a computer at the present time?

[chilledoutuk]

On that dv sample i tried a bunch of codecs at 2500kbps and the best was the mainconcept h.264 codec v2 here is a sample of that.

http://rapidshare.de/files/36664008/Copy_of_ga4.mpv.mp4.html

The inloop deblocking is off and with that on i believe any blocking still there will not be visable.

[foxyshadis]

AVC can be read on PSP, video iPod, Xbox with XBMC, some palms and pocketpcs, hd-dvd and bluray players (it remains to be seen what sort of compatibility they'll have with non-dvd-compliant h.264, since hd-dvd and bluray are extremely restrictive in some ways). However, all but the hd set-top players are too slow to decode at 2500 average, or even 2500 peak. And of course any home theater computer would work, such as a mac mini.

The xvid presets can be done in megui, virtualdub, or even henryhk's batch file, so don't feel you need megui for it.

Downloaded, but haven't had a chance to encode yet. Beautiful, though, and I see why it tortures codecs.

[sadie]

@HenryHo

Belatedly, tried to load your suggestions into script. Came back with virtualdub error: "Tdeint: YV12 and YUY2 data only". Source is the same Romeintro DV file I've been using all along.

@ChilledOut

Your compression of my clip is really quite good. The wave effect is barely noticeable. I compared it with my own xvid compression at 2500kb/s and smartdeinterlace. The blocking anomalies aside, I'm intrigued by the color aspects. Your clip has deeper tones slightly on the red side while mine is a tad flatter and perhaps neutral to cyan. Purely subjective of course. Mine also seems to duplicate the color rendition of the original. Anyway, yours pleases me more. I noticed also that the color space your file produces is YV12, adj whereas mine is YUY2 (as for most of the conversions I've been doing). Why is this? Does the difference lie with color spaces? What can I do about it? If you'd be so kind as to pass on your settings for that encode I could also do some tests when I get around to finding and installing the h264 codec. Thanks

@Foxy

Regarding h264 then it seems l'd be limited to viewing any avc file on a comp setup. My stand alone certainly can't handle it.

To put a new curve into all this, is it of the opinion of any of you all that perhaps I should forget about xvid altogether and simply put all my efforts from now on into avc/h264? What are the stakes, if any?

[sadie]

I'd like to correct what I posted for Chillout just a while back. I'd said his h264 clip encode had superior color saturation in YV12 adj mode compared to my xvid settings that outputted to YUY2. False. Opening and closing different versions of that clip in ffdshow induced the very opposite color space specs that I'd mentioned. Meaning: I could open Chillout's h264 file in YUY2 faded cyan mode while conversely opening my own in that deeper magenta YV12. This is absurd. How can the same file be opened in different colour spaces using the same codec simply by opening and closing the player successively. I've never seen anything like it. To qualify the problem, it seems to happen when I open multiple windows in zoomplayer, the first being YV12 the others YUY2. And as I still don't know the relevance of either space to the final result, it sometimes makes me want to chuck this video string theory business down the drain and long for my Kodachrome/Velvia days of not so long ago.

Ok, I managed to translate one of Teeg's profiles (the 58% quality one), and the same size encode produced no improvement whatsoever over my standard area+smartdeinterlace+smartsmoothHQ settings at 2500kb/s. So much for that.

[chilledoutuk]

HI there i think the reason your experiancing different colour spaces is that the hardware overlay which is normally yv12 normally can only be engaged once if you have a video open then that will be using the hardware overlay of your graphics card.

I would also point out that the overlay can have settings that differ from your normall display settings that only affect video that is played back using the hardware overlay.

Ok the settings i used avisynth plain mostly but i used toms motion compensated detinerlacing filter.

apart from that i used bassically every bell and whistle mainconcepts v2 encoder had.

I would however like to point out that i used x264 but it too had blocking issues on the sky also nero digital avc as well. I tried divx and xvid both had the same issues.

I suspect that mainconcepts h.264 encoder is filtering the chromatic change out as this product and as this product was primiraly targeted for content creators with uncompressed or dv source.

[*.mp4 guy]

The mainconcept encoder isn't doing anything special, I got similar results with X264 (http://download.yousendit.com/CF61AC9254FF1E79).

[sadie]

Chilled out, thanks for reply. I installed Mainconcept h264 v2.0 and my first attempt at encoding the 16 minute mother file from which the 10 second clip you worked on came resulted in a minor disaster. Image shake, large colored blocks breaking the picture every several seconds, etc. I used the standard default settings (H264 main, Pal, Direct Show import,1-pass, Program(vid+aud). Only modif was to 2500kb/s and deinterlacing specifying bottom field first. I also attempted to go the AAC route at first but twice at frame 3909 the encoder spat back some buffer error and stopped. When I shifted to mpeg layer 2 the encode proceeded normally. It would be immensely helpful if you could share a couple of your whistles and bells, or even an avs script. That is if it isn't a hardware problem. Thanks.

The YuY2 vs YV12 issue is still just a big YoYo to me. Today, your MP4 file plays Yv12 while my DV avis are showing YuY2, whether or not I load them individually or in multiple windows. Worse, since switching to Media player classic as default viewer Zoomplayer now warns me that Windows is unable to find the video file even as the video starts to play in the program! Further, ffdshow shows clip property bitrates in the range of 160000 to 300000 kbs/s and will list for ex. both xvid and ffdshow as the active filters for a clip (instead of one or other). I recently installed K-Lite codec pack....hmmm....They had me unload xvid and ffdshow before installing their own versions. Maybe I need to do a little digital dusting to get the house back in order. Any ideas on this.

And MP4guy, happy to hear there is more than solution, but for the time being the MC interface is one less complication to worry about.

If this thread has shifted from its xvid origins, apologies. I'd be happy to transfer it if I could.

[sadie]

Quick update to myself. The H264 file runs more or less well in Zoomplayer which is plugged to the avc format. Oddly, thought Mplayer classic, the default player, also runs ffdshow yet the playback sucks. What gives?

Regardless, problems persist since the playback although relatively 'clean' stutters greatly (somewhat like xvid does in certain encoded situations). And I'm still interested in Chillout's tweaks. Later all.

[foxyshadis]

Try turning YV12 off in ffdshow, and/or try switching zoom player's and mpc's colorspace. vmr7/9 renderless should do it, but it's also slower than overlay. (mpc: options->playback->output, zoomplayer: options->filter control, which is only in advanced mode) Do the level correction in ffdshow's levels filter instead, if non-overlay is giving you unscaled output.

As for ffdshow, I can't recommend anything but the latest tryout (http://ffdshow-tryout.sourceforge.net/), which is faster and more stable than the old version k-lite uses. Some recent h.264 tweaks are really speeding it up, probably won't be finished for another week or two.

My guess is that the stutter comes from not using overlay (overlay is much faster than VMR7/9 if you're at the edge of playbackability) in MPC. And that the mainconcept problem is due to interlaced encoding, possibly.

[sadie]

Foxy, as usual your interest is appreciated. But also as usual since I don't yet have a clear idea what the ramifications of these colour spaces are all about it's half useless to me. I mean when I'm told that DV is universally YV12 and a DV file is read by ffdshow as being YuY2 INPUT, the consternation goes beyond questions of inverting color spaces. Until I've got a working knowledge of this domain (and people have warned me against attempting to waste my time on it) fiddling with square plugs in round holes (even with enough force) will neither cut my intellectual curiosity nor solve the problem. Odd how for most subjects a 30-second google will put you in the lap of the knowledge maker of your sought after question. Well not so with the YvYuV gang. And neither so on this website. Maybe there's a dedicated sticky FAQ in DOOM, but I haven't found it and no one else has pointed me in that direction if there is one. Suggestion: Doom-doers, do one. I did find a forlorn site somewhere on the web of a repertoire listing some 50 color spaces to be found in the marketplace. And nearly all contained the letter Y. Perhaps because the letter is a homonym for the word 'why'. For there simply can't be a reasoned justification for it all. Or have I merely got a limited notion as to what chaos theory is all about. Repeating myself, yes, I wonder how we photographers managed to survive all these years with just two color universes to live by: Fuji and Kodak. I'd answer it's because the stress of our world was to make images that one couldn't post-process and paste. Your stress is competing with all those other color-space makers and their homologues to correct and compact images that may not even deserve the treatment. Pixels before Picasso.

The digital world has become one where there is no longer an exact reference point for anything. The notion of a perfectly exposed shot today resides in the collective makers of Photoshop and Premiere. The clickety clacker hasn't got a clue. I look at my own first experience with my new Panasonic of last year. Open the LCD on a bright summer day. Your subject translates into a diluted nearly invisible ghost image. Checking the viewfinder: darker,the contrast is there but you can't tell if the person is laughing or crying. Toss a coin and roll the cameras. Back home you capture your file to Pinnacle or its brothers and on the monitor you preview colors that look rich but a bit on the dark side....so dark that you even throw out a certain number of scenes as being too underexposed. You then take your burnt DVD to your old Trintron and you realize that the images you'd chucked were probably perfectly good after all. Too late. And then when finally you return to play the video off your computer monitor the images appear to have come full circle: washed out and soft all over again. Which leads you to go to Doom to find ways to bring back that image quality you can almost still remember from the time you initially shot the images. This is progress?

Well, a vast subject I could go on for light years with. But the venue here is, of course, inappropriate. Anyway, stuck as we are in this digital age, you Doom people are doing your part to make it a little less painful....sort of.

Foxy, I'm covered on the overlay issue 100%. So it can't be the source problem. And I've tried doing a couple of tests with mainconcepts, delaced or not, with no major difference in improvement. I think your phrase 'edge of playbackability' is at the heart of the problem. But what does that really mean?

Apologies again to the moderators for the short diatribe above. It won't happen again.

[foxyshadis]

Once avisynth.org comes back up, I can point you to a short explanation; fourcc.org is more of a reference but it's a good resource as wel. For the most part, YV12 can be upconverted to YUY2 or any other YUV colorspace with no loss of quality. It's the conversions to and from RGB that are problematic.

But the main problem is that some vieo cards and video card drivers or the renderer treat different colorspaces differently; for instance, one might convert yv12 to rgb in tv mode (16-235->0-255), while doing yuy2 in pc mode (0-255->0-255), which is where these over/undercontrast problems crop up, even on the same video. The main way to force them is to only output rgb, so the conversion is done in ffdshow, but you can also experiment with different yuv output colorspaces (which are faster) to see which work and which don't.

[henryho_hk]

Computer monitors and TVs always look different, even after the monitor is tuned. We cannot have the same material looking good on in both world. Most of the time, e.g. DVDs are tuned for TV and it's the PC software's job (and the PC user's too) to make them look "alright" on a PC monitor (brightness stretching, gamma, color and maybe also deinterlace, etc).

[chilledoutuk]

Bassically i used highprofile 5.1 with 4 referance frames with a framerate of 25fps i switched off deblocking whilst testing so that i can see whats going on more clearly.

here is a settings file i created for you it should allow you to encode just remember to set the deinterlace to the field order of your video. I used a avisynth filter.

http://www.sendspace.com/file/b469gf

may i ask what settings were used for the x264 encode as i encoded with settings that were still slower than mainconcept yet its quality was still infererior.

[sadie]

After a 24-hour turnaround at my country shack with slices of setting sunlight sandwiched between a thunderous sky and rain-soaked fields of green begging for the week wacker, I'm back at home plate.

Appreciate the responses inspite of my latest flake-off. It was hardly meant as a swipe at the Doom endeavor, or even less so at anyone in particular. Progress, at times, DOES make perfect. At other times, it...well...just demands more progress.

@Foxy and Henry

Your words 'rendered' 90% with me. But as usual there was something missing that prevented me from putting the whole picture together into something as smooth as margarine.

It's clear, I'm just going to have to buckle down to do the painful homework to get that color space light to shine on me. I'm also going to do a radical uninstall of untold numbers of recent programs and packs, do a little register cleaning and get back to plugging things in on a one-by-one basis. Am I wrong or has my old favorite Zoomplayer gone the route of bloatware from its initially light origins? Is Mplayer classic the player of choice these days?

@Chilledout

Thanks for info and link. Will check it out tomorrow. Straight off, your 5.1 profile differs from my 3.1 and I will also try to unload the default deblock option in MC H264 v2.0. By the way, are you using the same version of MC? The latest v2.1 apparently implements MP4 in output? Not that this would necessarily change the stutter issue at hand. The second part of your post, must be directed to Mp4guy. Otherwise, I don't understand it. I also would be interested in his reply. Good of you to help out on this.

[sadie]

Chilledout, I loaded your mef profile into MC H264, ran it, renamed the mpv it produced to mp4, received a Haali splitter error in mplayerc (saying it didn't recognized the media), but proceeded to play it anyway and the results were greatly inferior to what should have been the same file that you posted. The wave effect was pronounced and there were numerous blocks in the sky to boot. I can't possibly see what could go wrong here. My file size is slightly larger than yours 1.64mb vs 1.63mb. Why aren't they identical? Doing the encode with the standard H264 main profile, the quality was oddly slightly better (meaning still mediocre). Oh, one more thing: your file does not produce the Haali splitter error box before playing it. One other anomaly, the file I made cannot be played in continuous loop mode. After the first pass, Mplayer returns to the first scene and hangs/stutters forever on that opening frame. Again, this doesn't happen with the clip you made?!?

As a side note I d/l-ed mp4guy's test and, at least on my computer, the quality was nothing in comparison to yours (i.e. very wavy).

[*.mp4 guy]

may i ask what settings were used for the x264 encode as i encoded with settings that were still slower than mainconcept yet its quality was still infererior.
ME-Hex, Motion search precision-6, Key interval 240, min key interval 48, all high profile and bframe options, 3 mixed refs, no dct decimation enabled, no fast pskip enabled, max 3 bframes, bframe mode set to none, M4G smoothv1 was used as a cqm, deblocking was -2:-4

@Sadie
I thought what you meant by "wavy" was blocking/banding/pulsing artifacts on gradients, which the X264 encode has very little of. I suppose what you mean as "wavy" would be the minor smearing that was caused as a trade off of reducing blocking. Personally I find it less disturbing then blocking and flickering gradients, but if you don't I could try to trade more blocking for less smearing.

[sadie]

MP4guy, my problem is that which occurs in a blue sky and similar un-nuanced color gradients wherein you find a sort of rising or falling curtain that, yes, waves like a flag or an aurora borealis on fast forward. Without going to Greenland to verify this, consider it closest to what you and others have described as banding with blocking. And your video does have this on my machine. No, my issue is not one of smearing. Both your test and Chillout's are very good for the 'grounded' portions of the video and rate with the mpeg2 quality I'd made of that clip. In fact, I'd say your test is even a tad sharper and has less shimmering along certain lines. I can't do a head to head comparison because one will invariably be YV12 and the other YUY2. (Anyone feel free to elaborate as to why, technically, one cannot open two YV12 windows simultaneously). Still, Chillout's 'smoothed' out sky is noticeably superior to yours. But to repeat myself, using Chillout's formula myself I got very mediocre results as well. So either Chillout has left out something in the parameters he communicated or I've got a hardware/configuration issue at my end. Simple as that. I just don't which it is.

Suggestion Mp4guy, I'm going to try this weekend to run through your x264 scenario, if I can figure it out, and report back to you if it gives same, better or worse results than your link. Why don't you try to run Chillout's MainConcept mef file and let us know the outcome. That is, if following his recipe gives you that 'smooth' sky then it must be a playback problem from my end. If 'blockywaves' then we might be able to say Chillout is holding on to a quality-control trump card somewhere. Thanks for continuing interest.

[henryho_hk]

sadie, your clip is indeed very grainy. The curtain artifact remains there even when I encode your clip as grayscale (gray curtain this time). Graininess plus fading seems tough for MPEG4 ASP codecs. If you want to stay with XviD, I suggest you to use strong denoisers, for example, degrainmedian(mode=0).degrainmedian(mode=1), FFT3DFilter(sigma=2.5) or even FFT3DFilter(sigma=3). I made another sample clip at http://www.sendspace.com/file/p31t8j. The sample clip worths about 4300kbps and I think it is acceptable for a particular scene within a long(er) footage. Let's see if the denoise filter has brought the curtain artifacts down to an acceptable level.

BTW, I use the free Cedocida DV codec and configure the output as MPEG2 4:2:0 Interlaced. And I have applied the FFT3DFilter(sigma=2.5) denoiser in this sample clip.

Edit 1: On further testing, it seems that Jawor's 1CD matrix can also handle the clip very well (maybe due to its high smoothening effect). It is available at http://www-users.mat.uni.torun.pl/~jawor/matrix/index.html.

Edit 2: Try FFT3DFilter(sigma=3,sharpen=0.3) for denoising+sharpening

[chilledoutuk]

just a quick comment with mainconcept you need to encode to elementary video and then name the file .264 then use the latest YAMB 2.0 beta to mux that stream into a mp4 container.

Also i am using tomsmocomp deinterlacer on your dv video are you as well?
If your using the deinterlacer built into mainconcept that might explain why your getting different results.
Its possible the verticle filter on tomsmocomp is smoothing a little.

[*.mp4 guy]

Sadie, I think you are having a problem with PC/TV levels, the clip I made was encoded at tv levels, and looks fine when viewed on a tv, or with tv levels on a computer monitor, but it indeed has bad banding when it is convertted to pc levels during playback. This is because the footage already has minor banding in it, and when additional compression and color conversions are done the banding is inevitably made worse. compression also kills the noise, which was hiding the banding to some extent in the original footage.

[sadie]

Ok, fellows, I haven't looked into your recent suggestions/comments fully as yet. However, yesterday I backtracked a bit and tried to deal with a few mpeg2 encoders to see how they compared/tested with the xvid/h264 problematic. Well, first off I've got to say if xvid or h264 could do menu titles and standalone players were up to speed I'd just as soon tie a 50kg weight around mpeg2's neck and drop it into the Mariana Trench.

CCE vs Procoder vs Pinnacle built-in, the differences in an 'easy' encode are to my eyes negligeable. So nil that it's not worth pulling the darn thing out of the Pinnacle box and running it through an additional two programs. That said, I've got a slight preference for Procoder. Is it the contrast/saturation? A little less noise? I really can't put my finger on it.

As for problem areas with such things as those wavy curtains in monochrome land, none of them do well at all. Yes, in fact all of the encoders handle the anomaly worse than any of your attempts thus far. Henry hit the nail on the head: it just seems that noisy non-gradient color + fade transitions are a recipe for snapping the encoders at their weakest link.

Now, I've partly cleared up the YV12, YUY2 affair. I falsely attributed color playback variations to space distinctions. It ain't necessarily so. In fact, I noticed the 'new' mpeg2 filter settings under ffdshow playback that allows me to a) uncheck the 'enable planar YUV types' and b) adjust the TV>PC settings. Both allowed me to verify that with two windows open the second window regardless if the first one is YV or YUY will exhibit the color degradation and contrast. Also with two windows open the wavy curtain problem is WORSE in some sequences. But I doubt it's a color space issue as such; perhaps an overlay issue (although i've no idea what the mechanism could be). In short I at least do not have a fundamental problem connected with color spaces.

I'll go on with my tests.

@Chillout, no I did not use a delace filter. As I said, I just plugged in your mef profile. I presume you simply loaded an avs file into mainconcept then? Did your script include anything other than tomsmocomp? Thanks

@Mp4guy, When you refer to PV/TV levels are you speaking about the same thing as Mug Funky higher up the thread. If so, I realize that DVD's are 'tuned' for TV (in Henry's words) which is why the debanding tweak in ffdshow is a welcome aid. But we're trying to deal with the perhaps impossible task of eliminating it on the computer. Further, as concerns H264 I am not setup/wired for a TV-out connection. So, apart from investing in one of the very newest DVD players that will accept avc, I'm limited to viewing avc/mp4 on the computer monitor.

[Teemo]

Sadie, would you mind uploading the original source again. The storage at Rapid is full and your clip has been kicked....
.savefile. seem to be working (link at previos page).

I find chilled's clip looks nicer than .mp4's
.mp4's clip has more visible artifacts that catch the eye.
On the other hand I find it is sharper and has more detail in shadow areas.

Im absolutely new to avc coding and have problems with some sources. Encoded "Hero" hdtv, which is in some places very noisy. Ran i on xvid and x264 at 4000kbps, both at default settings. Prefer the xvid of the two, as x264 did not succeed in masking the noise, whereas xvid kept it.
x264 transformed the noise to what could be the same as Sadies curtains. Visible blocks dancing around.
Any suggestion to how we get rid of the curtain/blocks is appreciated :-)

In another source, a .ts file from satellite (Sleepy Hollow) is difficult. It's dark with lots of fog and noise. Here I found the NRS filter (vDub) useful. x264 performed a little better than xvid with this source.

...and in the meantime i'll keep on reading/testing/reading/testing...etc. No pain no gain as they say.

[Teemo]

...oh, forgot to drop a question:

Does the new x264.exe have significant advantages over the (outdated) vfw release used with vDub ?

Yes I'm one of these old dogs that still use the goodol' Vdub. Main reason is that I refuse to downgrade my ultrastable w2000/sp2 box with sp3, which is required to install .NET, which is required to install all the goodies for x264. Sigh.
Fortunately avisynth and a cmd-box works fine with the new x264.
Maybe it's time to step up to Linux. Does it have the right tools to create avc's ?

[sadie]

Hello, Teemo, welcome to the thread. Delighted to hear you are having similar problems as I. As I have 1 day's experience on H264 and 0 on x264 (hopefully tomorrow) I'm perhaps not the one to advise you on much. Especially since your angle seems to be HD commercial sources and not DV.

I concur exactly with your analysis re/ Chillout and Mp4guy's respective clips. Today I managed to get another partial handle on at least Chillout's attack on the problem. First, I discovered the massive periodic blocking I had in my tests, were due to nothing less than being overzealous and fatigued so that the avc file was running on 8000kb/s rather than 2500. Duh.

That waft of brainlessness set aside, I found that with at least H264 (I've yet to follow Henry's xvid tips) it's one evil or the other. On the one hand you can have Chillout's mainconcept profile incl. tomsmocomp delace filter (0,5,1) which will smooth out the curtains yet sacrifice, in the end, a fair amount of sharpness. Or you can go the non-deinterlaced route and get some real sparkle for your money, but have to tolerate those exacerabating waves. It's one or the other. Could be that on a TV, the interlaced settings might turn out to be the preferred way to go. As for the computer, the gradfun filter is a viable workaround. But it just doesn't seem like the most elegant solution. Temporalsoften and kerneldeint don't even scratch the problem.

And then there are different sorts of banding. I've found it is somewhat easier to correct in a horizontal pan or a reverse zoom than a static shot with fades at either end. Teemo , tomorrow I'll attempt to repost that clip (and perhaps a couple of others) and you can attempt to see for yourself.

Anyway, keep up the pain.

[chilledoutuk]

Hi there i decided to play around a little more i feel that 2500 is a little to low a bitrate for the detail in the clip you uploaded so i decided to use lanczos filter to reduce the resolution to 640x480.

I find the results more pleasing so here is what i encoded with mainconcept v2.0

http://www.filefactory.com/file/0f8f80/

this one is direct dv into mainconcepts h.264 encoder using the built in deinterlacing with the normal settings but this time at full 720x576 resolution.

http://www.filefactory.com/file/d26e5a/

Oh here is the dv source that someone was asking for.

http://www.filefactory.com/file/fba80b/

[Teemo]

Got the clips and the source now, thanks Chillout.

Sadie, no need to upload again ;-)
Actually I have mostly done xvids of captured analog video; 300+ musicvids and 35 concerts. Stuff that needed deinterlacing and use of filters, so it should be close to your recordings except I think yours are cleaner than my caps.

Its getting late, but I will play around with the bits tomorrow.

[Teemo]

Well... not an easy one, that blue sky.
Actually the source has some of the same unwanted effect. If you play it with ffdshow, try to set Levels->Input to Low=80 High=170 you can easy see it (vdub will do as well).
Is there a way to make the encoder less sensitive to blue or do more deblocking on blue (before encoding) ? Would probably have other negative sideeffects though.

[henryho_hk]

It's not the blue. You will see the same effect when the clip is changed to greyscale using tweak(sat=0).

[Didée]

For testing, a possibility with XviD would be to assign zones to the sections with those difficult fades, and configure the zones to be coded with I-frames only ...

... but only if this long-standing zone feature suggestion would have been realized, which it never had. :)

[*.mp4 guy]

I want to make something clear, the codecs aren't the problem here, the source clip has the problem, and the codecs are only doing the best they can to preserve the original clip characteristics.

Here is a contrast enhanced clip that shows the problem (http://download.yousendit.com/55DEB57F4B37C3A0)
and here is a hue enhanced clip (note that saturation wasn't changed) (http://download.yousendit.com/2BF54CD80C9BC13C)

It looks to me like the original dv source has overcompressed dc coeficients, especialy since the grain is preserved perfectly.

[sadie]

Ok, let's see, I've finally caught up on a few things.

@Henryhk

Your well-meant suggestions, just don't quite bear up I'm afraid. I did several variations of degrainmedian or ff3dfilter with tdeint in vdubmod and they came off with such lovely well-defined drapes that I'm now almost tempted to take out a patent and market them as Hollywood special effects. At sigma 3, the latter filter starts to spread the goo a bit much. But you're quite right, at about 4500kb/s you start to get a nearly decent smoothing out of those bands. Problem is, as I'd stated earlier mpg4 is only really worth it to me if I can get file sizes of 1/3 or less compared to DVD. And here we're roughly at 60%. Thanks for Jawor's matrix, but isn't that meant for encodes of rather low biterate?

@Chillout

Haven't had chance to see your latest test. When I finish sanding the stairs it will be pleasant to come back and see it. As for your first test, I've come close to but never quite matched your quality. Are you doing the straight (0,5,1) tomsmocomp settings and loading the avs script straight into MCh264 like one would in vdubmod? I tried it with TFF as well and didn't seem to suffer in playback quality. Odd. You suggested renaming to .264 and muxing with your yamb program. I did that and the file plays back like a Mr. Phelps self-destruct tape. In 6 seconds, the pixels run amuck from a nice clear image to maybe 2 square pixels at the close. Never seen anything like it. I used yamb v 1.6. Why not just rename the extension .mpg and you're home free?

Deblocking in mainconcepts seems irrelevant to our problem. Also there is not a substantial difference between the 3.1 main and 5.1 high profiles in MCh264. On the other hand, regardless of which profile, turning on the vertical component of tom's delace filter is crucial to getting results. So I ask you all if there is another filter out there that incorporates the principle of that component into another tool (smoothers, denoisers, etc). Get back to you on your resize test.

@MP4guy

"compression also kills the noise, which was hiding the banding to some extent in the original footage". I'd thought banding originated because the lack of noise prevented the encoder from 'grabbing' on to anything in those flat gradient-less zones. I tried a nominal encode or two with x264 in vdubmod making the few tweaks of yours that I could: fair results using area deinterlace/smartdeinterlace combo. I'll have to make a go at megui to use your settings.

@teemo

I uploaded three file for you before seeing that Chilledout had so kindly obliged you. See http://www.sendspace.com/file/p2fzed
http://www.sendspace.com/file/zyyd4c
http://www.sendspace.com/file/0jz1y1

Each clip present different degrees of the banding problem according to the way they were shot. Although the fixed image is the worst. Go figure.

[foxyshadis]

I want to make something clear, the codecs aren't the problem here, the source clip has the problem, and the codecs are only doing the best they can to preserve the original clip characteristics.

Here is a contrast enhanced clip that shows the problem (http://download.yousendit.com/55DEB57F4B37C3A0)
and here is a hue enhanced clip (note that saturation wasn't changed) (http://download.yousendit.com/2BF54CD80C9BC13C)

It looks to me like the original dv source has overcompressed dc coeficients, especialy since the grain is preserved perfectly.

Since that was the impression I got as well, I was working on a script to add a very strong temporalsoften to the flatter areas, post-denoise, although that wouldn't work so well on the fades. My hope was that it would at least cover up the problem a bit, if not eliminate it. (I fell asleep before getting far though.)

[sadie]

Mp4guy, it's never been denied that there was a source problem. And while you're on that subject which in a way is more worrisome , what tells you the file is 'overcompressed'? What would a 'normal mini-DV show you? That DV file except for a single render (according to Pinnacle and its theoretical 'smart renderer') is otherwise straight out of the field. Going back then to one of my earlier suppositions, is the camera defective; is there a preferred DV codec to be used that avoids these issues? Or perhaps this is standard for MiniDV cameras. I think Mug Funky gave his opinion on this. Anyone else dare to stab it? Any camcorder users amongst you?

To Chillout, once again your tests are a mixed blessing. Both your links are quite decent. Yet, when I attempt the same 'standard' settings the playback manifests extreme blockiness right in the middle of the clip. I cannot imagine what we're doing differently. If only you could preserve a detailed log. Can anybody else reproduce Chillout's encode quality?

[henryho_hk]

at about 4500kb/s you start to get a nearly decent smoothing out of those bands.

4500kbps or more may be required for this particular scene but definitely not the whole movie. With a proper 2-pass encoding (maybe some manual zone tuning), the codec will give difficult scenes more bits and easy scenes less, and as a whole it may be something like 2500kbps (a user specified parameter in 2-pass mode). It all depends on the composition of your whole movie.

Thanks for Jawor's matrix, but isn't that meant for encodes of rather low biterate?

"rather low bitrate" is a relative concept. For an easy scene, it may mean 300kbps. For a difficult scene, it may mean 3000kbps.

[*.mp4 guy]

Mp4guy, it's never been denied that there was a source problem. And while you're on that subject which in a way is more worrisome , what tells you the file is 'overcompressed'? What would a 'normal mini-DV show you? That DV file except for a single render (according to Pinnacle and its theoretical 'smart renderer') is otherwise straight out of the field. Going back then to one of my earlier suppositions, is the camera defective; is there a preferred DV codec to be used that avoids these issues? Or perhaps this is standard for MiniDV cameras. I think Mug Funky gave his opinion on this. Anyone else dare to stab it? Any camcorder users amongst you?


I have a camcorder, and while it doesn't have that specific problem it has others which are worse, so you will likely just have to deal with the problem unless you are willing to buy a much more expensive camcorder; I very much doubt that the one you have is defective. A different mini dv camera (I'm taking mine as an example) would have less details, but would also have less blocking/banding artefacts. DV is a standard, so it is used in all dv cameras (obvious, I know), but there are many implementations of the standard, and the one your camera uses evidently overcompresses dc coeficients, especialy chroma dc coeficients, though it is otherwise quite good, in my last post I was just trying to give people a more clear idea of the problem that needs to be fixed to completely avoid banding.