WM9 encoding ugly as soon as I resize

[BangoO]

Hi there ;)

I am encoding HDTV 1080i material, with the following script.
Instead of explaining the problem, 2 captures will show you everything...

Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\mpeg2dec3.dll")
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\TIVTC\TIVTC.dll")
mpeg2source("D:\test.d2v")
cropbottom(8)
TFM()
Decimate()
LanczosResize(1280,720)

Here is a result frame, look at how ugly it is in the left green part.
http://perso.wanadoo.fr/bangoo/720p.jpg

Now, this one is the same script, without the resize (the picture has been resized after the encoding so that it can be compared to the first one):
http://perso.wanadoo.fr/bangoo/1080p.jpg

In this one, the green left part is good.
Whatever resize values I give, whatever resizer I use (bicubic, lanczos, simple), as soon as I resize the green left part gets ugly...

Of course, before encoding it in WM9, it's good (when you look at the VDubMod window).

So... without a resize, it's good before and after encoding.
With the resize, it's good before the encoding, but not after...

Any idea about what could be wrong here ?
Thx ! ;)

[BangoO]

I forgot to say that I encode it in VBR 90 quality.
I just tried CBR 8mbps, and the problem is still there, but a LOT less !

[Mug Funky]

that's peculiar. how do other encoders handle this? i suppose xvid or similar isn't an option on this stuff?

i don't know enough about WMV to be able to say why it was going all blocky like that (actually, i'm on a 16-bit display at the moment, so i barely saw the artefacts without having to look really hard).

[BangoO]

XVid gives me something horrible on this frame, resize or no resize :)

[Didée]

Some slight denoising should help much. If there is noise, but not enough bitrate available to reproduce it, then this is exactly what you'll get on flat surfaces and gradients.
Plus, if noise is there, then it usually gets more ugly through resizing (unless a very soft resizing is used, or one resizes <50%).

Try e.g. MergeLuma(last.TemporalSoften(2,9,5,23,2),last,0.49) and see if it gets better.

Alternatively, you could simply force WMV's postprocessing on playback time ... I'm sure there will be no more blocking visible :D

[BangoO]

It's a lot worse if I add you MergeLuma...
I really don't think that it's a problem of noise, otherwise it would be there in 1080p as well.

Could it be a colorspace conversion during the resize ?

[Didée]

Originally posted by BangoO
Could it be a colorspace conversion during the resize ?
A what during the resize?? Perhaps a resizer is internally changing the polarity of earth's magnetic field, dunno. But for sure it does not do a colorspace conversion.

Without something to try, I'd recommend to just encode at full res. If it's fine that way - fine. :)

[BangoO]

The point is to reduce the file size, which is mainly done with the resize, so I have to keep it.
Anyway, I would like to understand what's happening...

PS: by colorspace conversion, I meant YV12 to YUY2 or anything else, but I guess you understood ;)

[Didée]

Sure I understood. And sure, such isn't taking place during resizing.

But oh, while looking at gamma-corrected pictures (to be able to see something at all), I totally forgot that those problematic areas are pretty dark indeed. This brings us back to the "dark blocking issue", which already was discussed many times on this board. Suggestions were plenty ... cleaning noise, adding noise, using lumafilter(-2).unfilter(5,5), ... whatever. General conclusion there was none. Sometimes this helps, sometimes that, ...

Assuming that the scene you're refering to is rather static (?), just for testing try a simple temporalsoften(3,20,20,40,2). IF this helps, one would make a mask to restrict the strong temporal smoothing to very dark areas only.
If it doesn't improve on the static parts, it's time to try the opposite: adding noise through the lumafilter/unfilter combo, or through Blockbuster.

[BangoO]

temporalsoften(3,20,20,40,2) really remove the noise (which I want to keep), and the result is even worse than before.

The scene is static.
I understand that this could be a dark scene common problem.
But this would not explain why there is no problem as long as I don't resize (the scene is still dark when I don't resize :)).

Thx for your help ;)

[Ark]

If you look carefully, you'll see that the whole image seems simply more compressed, even bright areas are slightly more *blocky* (well, actually is only some compression-noise introduced), but you have to look really close...

I think that you're simply using a different (and slightly worse) compression ratio. At what bitrate are you aiming for the two encodes?

[BangoO]

I'm using the VBR 1 pass quality encoding, set to 90.
On the 720p, it's a 3770kbps average (so says VDubMod in File Info).
On the 1080p, it's a 9300kbps average.

I tried encoding in CBR, with the same bitrate as the 1080p, it's a lot better, but not as good as 1080p.
Using the same bitrate, it should be better at 720p than at 1080p, as each frame is smaller... I don't get it :)

[SeeMoreDigital]

Originally posted by BangoO
I'm using the VBR 1 pass quality encoding, set to 90.
On the 720p, it's a 3770kbps average (so says VDubMod in File Info).
On the 1080p, it's a 9300kbps average.

I tried encoding in CBR, with the same bitrate as the 1080p, it's a lot better, but not as good as 1080p.
Using the same bitrate, it should be better at 720p than at 1080p, as each frame is smaller... I don't get it :) Hi BangoO,

Can you confirm... Is the source 1080p or 1080i?

If the 1080 source is interlaced, are you de-interlacing it before converting to 720p?

If you are de-interlacing, how are you doing it. And how many frames per second are you encoding to?

Personally, I think 1pass encoding (resulting in 3770Kbps) is too low a bit-rate for HD encoding. If this is the sort of bit-rate you are aiming for, 2passes would be better.


Cheers

[kassandro]

Originally posted by SeeMoreDigital
Personally, I think 1pass encoding (resulting in 3770Kbps) is too low a bit-rate for HD encoding. If this is the sort of bit-rate you are aiming for, 2passes would be better.

You seem to believe that two pass encoding is vastly superior over one pass VBR (not CBR) encoding. If the resulting file size is the same, then there is not much of a quality gain. The main advantage of two pass encoding is that you can achieve a fixed file size, but you pay a high price if the encoder is as slow as the WM9 encoder (really unusable for real work).

[BangoO]

Originally posted by SeeMoreDigital
Hi BangoO,

Can you confirm... Is the source 1080p or 1080i?

If the 1080 source is interlaced, are you de-interlacing it before converting to 720p?

If you are de-interlacing, how are you doing it. And how many frames per second are you encoding to?

Personally, I think 1pass encoding (resulting in 3770Kbps) is too low a bit-rate for HD encoding. If this is the sort of bit-rate you are aiming for, 2passes would be better.
Hi SeeMoreDigital,

The source is FILM 1080i, and the script used to convert it to 720p 23.976fps is in the 1st post.
I am deinterlacing it (IVTC to get 23.976fps), then resizing it.
I tried 2 pass, it does not change anything.

[Slogra]

I'd say blame the LCD monitors with their poor contrast and high brightness. It looks great on my CRT monitor, very close to black like it should.

[BangoO]

Originally posted by Slogra
I'd say blame the LCD monitors with their poor contrast and high brightness. It looks great on my CRT monitor, very close to black like it should.
I can't see it either on my CRT projector, I only see it on my LCD monitor :)
I would like to understand it anyway, because sometimes I can see it a bit on brighter scenes too...

[Soulhunter]

Originally posted by BangoO

...I only see it on my LCD monitor :)
http://img71.exs.cx/img71/1741/match8oe.gif


Bye

[SeeMoreDigital]

Originally posted by BangoO
The source is FILM 1080i, and the script used to convert it to 720p 23.976fps is in the 1st post. Thanks... I only asked because you kept mentioning 1080p ;)

Or have you been trying to generate 1080p/23.976fps encodes from the 1080i/29.970fps source too?

I have to say, the 720p image looks very good indeed on my high-def plasma screen!

Are you able to provide a short encode?


Cheers

[BangoO]

I encoded it a 720p and 1080p, both in VBR and quality 90.
The problem only appears on the 720p...
In the 1st psot, the 1st picture is the 720p encoding, the 2nd picture is the 1080p encoding.
That's the question of this topic... why does it look crappy as soon as I resize ?

[BangoO]

Soulhunter, calibrating is not the issue here...

[Seed]

I am not familiar with all this IVTC stuff (I'm from PAL land), so what I say below may not apply.

I seldom resize vertically. To do this on interlaced footage can produce quite some artefacts/blockiness. There are several methods proposed that might minimize artefacts/blockiness:

http://forum.doom9.org/showthread.php?s=&threadid=90950
http://forum.doom9.org/showthread.php?s=&threadid=88116

I found in a few cases (height --> height/2 or some other simple ratio), resizing on separate fields then deinterlacing may be better than deinterlacing then resizing:

SeparateFields --> SelectEven/Odd(), resize --> Interleave --> Weave --> deinterlace

Just experiment.

[BangoO]

Thx seed, I will give it a shot.
I was always "afraid" of doing anything before the IVTC process, as it compares frames to do is job...

[BangoO]

I tried this:

Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\mpeg2dec3.dll")
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\TIVTC\TIVTC.dll")
mpeg2source("D:\test.d2v")
cropbottom(8)
AssumeTFF()
SeparateFields()
top=LanczosResize( selecteven(last), 1280, 360)
bottom=LanczosResize(selectodd(last), 1280, 360)
interleave(top,bottom)
weave()
TFM()
Decimate()
I get the exact same bad result :)

[Soulhunter]

Have you tried other "non mod 16" resolutions ???

Coz 720 is mod 16 but 1080 not... :\


Bye

[BangoO]

Yes, I even get the problem with 1280*1080 and 1920*720...
It's really as soon as I resize, whatever I resize to (except from 1920*1080, of course :D).

[SeeMoreDigital]

Originally posted by Soulhunter
Have you tried other "non mod 16" resolutions ???

Coz 720 is mod 16 but 1080 not... :\ I think you'll find the 1080i Mpeg2 source actually contains 1088 pixels!


Cheers

[Soulhunter]

Originally posted by SeeMoreDigital

I think you'll find the 1080i Mpeg2 source actually contains 1088 pixels!

Hrm, is this valid for all 1920* Mpeg2 sources ???

Coz I have seen both, 1080 and 1088... :\


Bye

[Seed]

Originally posted by BangoO
Yes, I even get the problem with 1280*1080...
It's really as soon as I resize, whatever I resize to.

You mean that you have tried LanczosResize(1280,1080), and encoded the resultant (same height) avi clip to WM9?

My understanding is LanczosResize will sharpen the picture, and so even if you LanczosResize to the same width and height, there will be some changes in luma and chroma of adjacent pixels. These subtle changes will be amplified on encoding to the lossy WM9. These amplified changes will again be exaggerated (perhaps wrongly) on certain LCD screen.

(If there is any consolation, I fail to see any "how ugly it is in the left green part" thing on 2 well calibrated monitors of mine: one quite high end CRT monitor, and one DVI-input Samsung 172T LCD with very good contrast. Bear in mind some LCD monitors do not give 16 millions colours)

Try PointResize(1280,1080) ( or SeparateFields then PointResize(1280,540) each field). I would think that this will not give the same blockiness as given by LanczosResize(1280,1080). If this is the case you then may try bilinearresize(1280, 720). As stated in Avisynth doc, bilinearresize maybe better when down-sizing.

[BangoO]

Originally posted by Soulhunter
Hrm, is this valid for all Mpeg2 sources ???
Coz I have seen both, 1080 and 1088... :\
All the 1080i sources are in fact 1088 (in order to be mod 16).
That's why my scripts begin with:
CropBottom(8)

[BangoO]

Originally posted by Seed
You mean that you have tried LanczosResize(1280,1080), and encoded the resultant (same height) avi clip to WM9?
Yes.

I already tried bilinear and bicubic, the result is the same.

I understand that it is not visible on every display.
As I said, I can't see it on my CRT projector (which is the one I care about).
But this one is just an example that I had on-hand, I've seen this problem on brighter seen, and then it is visible.

By the way, I tried 1920*720 and 1280*1080 because avisynth's documentation says:
AviSynth has completely seperate vertical and horizontal resizers. If the input is the same as the output on one axis, that resizer will be skipped.
Which one is called first, is determined by which one has the smallest downscale ratio. This is done to preserve maximum quality, so the 2nd resizer has the best possible picture to work with.

PointResize(1280,1080) gives the same result :)

[Soulhunter]

Originally posted by BangoO

All the 1080i sources are in fact 1088 (in order to be mod 16).
That's why my scripts begin with:
CropBottom(8)

Thought its stored as 1080 but gets resized to 1088 while playback...

Or was it the other way around (1088 -> 1080) ???

Gah, I hate this 1080i/p stuff... >.<


Bye

[Seed]

Originally posted by BangoO
PointResize(1280,1080) gives the same result :)

I assume that your second (unresized) picture are produced from the following script?

Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\mpeg2dec3.dll")
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\TIVTC\TIVTC.dll")
mpeg2source("D:\test.d2v")
cropbottom(8)
TFM()
Decimate()

To really isolate the cause of the problem to ONLY resizing, try encode (at SAME bitrate) to WM9 the following clip:

Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\mpeg2dec3.dll")
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\TIVTC\TIVTC.dll")
mpeg2source("D:\test.d2v")
cropbottom(8)
AssumeTFF()
SeparateFields()
top = PointResize (selecteven(last), 1920, 540)
bottom = PointResize (selectodd(last), 1920, 540)
interleave(top,bottom)
weave()
TFM()
Decimate()

If this still gives blockiness, then we can conclude that no matter what resizers you choose, AND EVEN when "resize" to original width and height, resizing will introduce subtle luma changes in interlaced footage that will show up as blockiness in highly compressed lossy WM9.

And if so, I think a vertically smoothing spatial smoother might be most helpful (e.g. blur(0, 0.5) or unfilter(0, -5), to be applied after deinterlacing.

[Leak]

Originally posted by Seed
TFM()
Decimate()

Just curious - is there a reason you use Decimate (from Donald's Decomb package) here instead of TDecimate from TIVTC?

np: Markus Guentner - Chrom (Regensburg Remixe)

[BangoO]

Originally posted by Leak
Just curious - is there a reason you use Decimate (from Donald's Decomb package) here instead of TDecimate from TIVTC?
Yes, they both give the exact same result on this type of material, but Decimate is a LOT faster than TDecimate.

[BangoO]

Originally posted by Seed
I assume that your second (unresized) picture are produced from the following script?

Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\mpeg2dec3.dll")
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\TIVTC\TIVTC.dll")
mpeg2source("D:\test.d2v")
cropbottom(8)
TFM()
Decimate()
Yes.

Originally posted by Seed
If this still gives blockiness, then we can conclude that no matter what resizers you choose, AND EVEN when "resize" to original width and height, resizing will introduce subtle luma changes in interlaced footage that will show up as blockiness in highly compressed lossy WM9.
As I said before, this is what can be found in the avisynth's documentation:
AviSynth has completely seperate vertical and horizontal resizers. If the input is the same as the output on one axis, that resizer will be skipped.
Which one is called first, is determined by which one has the smallest downscale ratio. This is done to preserve maximum quality, so the 2nd resizer has the best possible picture to work with.

Resizing to the original size does not do anything.
I verified this when I tried your script (though I assumed you meant 1920*540 instead of 1280*540), and the result is good, just like if I don't resize at all.
Of course, if I use you script as is (with the 1280*540), the result is crap.

PS: adding blur(0, 0.5) doesn't change anything.

[Seed]

So you have tried "resizing" to original width and height (yes, I do mean to write 1920*540 instead of 1280*540).


Originally posted by BangoO
Yes, I even get the problem with 1280*1080 and 1920*720...
It's really as soon as I resize, whatever I resize to (except from 1920*1080, of course :D).

I guess I should have got it earlier if it were "(except to 1920*1080, of course :D).

I conclude what I understand so far:
1) The resizers are doing a proper job. When resizing in either one (width/height) or both dimensions, there will surely be changes in luma differences between adjacent pixels. (Probably more so vertically for interlaced footage).

2) The resizers are doing a good job, at least perceptually. Because the resultant avi clips are good when viewed on Vdub.

3) The WM9 encoding machine is not doing a very good job even when given relatively high bitrate (downsized clip given proportionately higher bitrate than the original clip). It translates mathematically distinguishable but perceptually undetected luma difference into slight blockiness seen on certain displays.

I wonder if mpeg2 or other codecs give the same problems ( when given relatively high bitrate - downsized clip given proportionately higher bitrate than the original clip).

4) Perhaps ultimately the solution, if you want WM9, is what was already suggssted by our in-house filtering guru Didee, to try various smoothers after deinterlacing.

PS: Try smoothing in horizontal dimension as well: unfilter(-6, -10) etc

[Seed]

Maybe Avisynth gurus/developers can answer this. When resize horizontally (100, 1) to say, (70, 1), does the resizers divide the 100 horizontal pixels into 10 groups of 10 pixels each, then individually resize each group into a width of 7 pixels and finally string together the 10 groups of 7 pixels into the new 70 pixels?

If so, correct me if I'm wrong, isn't it possible that there will be a little more luma difference across the boundaries of adjacent "groups" than intra-group pixels. E.g

In original clip:
luma of pixel #10 -luma of pixel #9
= luma of pixel #12 - luma of pixel #11
= luma of pixel #11 - luma of pixel #10

But in the resized clip:
luma of pixel #7 - luma of pixel #6
= luma of pixel #9 - luma of pixel #8, but
< luma of pixel #8 - luma of pixel #7 (Boundary of 2 adjacent groups)

And WM9 encoder may then cause blockiness upon these boundaries.

[BangoO]

Originally posted by Seed
I guess I should have got it earlier if it were "(except to 1920*1080, of course :D).
Oops :p

[scharfis_brain]

another idea:
what if WMVs postprocessing is too CPU hungry to be applied with 1080 lines stuff?
while the CPU power is high enough to postprocess 720p?

[Seed]

Perhaps FuPP's HybridFuPP 0.92b (26th March 2005) comes to rescue just in time? :)

http://forum.doom9.org/showthread.php?s=&threadid=92061

[BangoO]

Originally posted by scharfis_brain
another idea:
what if WMVs postprocessing is too CPU hungry to be applied with 1080 lines stuff?
while the CPU power is high enough to postprocess 720p?
Hmmm... the result is good at 1080p, and not good at 720p.

[BangoO]

Originally posted by Seed
Perhaps FuPP's HybridFuPP 0.92b (26th March 2005) comes to rescue just in time? :)

http://forum.doom9.org/showthread.php?s=&threadid=92061
Thx, I'll give it a shot :)

[scharfis_brain]

Originally posted by BangoO
Hmmm... the result is good at 1080p, and not good at 720p.

It maybe is smoothed out by heavy PP!

[Didée]

Catch. PP could be guilty, too. Time to quote it again:

[...]
To that end, here is a way to tune the post-filter in WMV9 decoder to get sharper images (but with possibly more compression artifacts). Use regedit to find the following key:

\HKEY_CURRENT_USER\Software\Microsoft\Scrunch\Force Post Process Mode

This is a DWORD value. The valid value is from 0 to 4, 0 being turning off post filter and 4 being the strongest. If the value is outside this range, the decoder will use the automatic mode to decide the post filter strength.

By setting the appropriate value, you should be able to easily trade off the two parameters stated above. Of course, this also changes the equation drastically wrt to observations made in the codec tests.

[BangoO]

It was set to 3, I tried 0 and got the same result.

Thx anyway fot trying to find the solution, guys ;)