HC "blocky" in really dark zones

[Fr4nz]

Hi all.

I've noticed that in really dark zones HC (I use version 0.23) tends to be "blocky", while CCE-SP mantains those zones similar to the original picture.
The settings I use with HC are these:


*PROFILE BEST
*INFILE E:\WORK\D2VAVS\V01000100001002.avs
*OUTFILE E:\WORK\D2VAVS\V01000100001002.m2v
*BITRATE 4564
*MAXBITRATE 8246
*WAIT 0
*CUSTOMMATRIX
08 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83

16 17 18 19 20 21 22 23
17 18 19 20 21 22 23 24
18 19 20 21 22 23 24 25
19 20 21 22 23 24 26 27
20 21 22 23 25 26 27 28
21 22 23 24 26 27 28 30
22 23 24 26 27 28 30 31
23 24 25 27 28 30 31 33
*BIAS 20
*LUMGAIN 1
*AQ 1
*PRIORITY IDLE
*ASPECT 16:9
*PROGRESSIVE


So, the question is: what is the culprit of the "blockiness" in dark zones? May I change something in the settings in order to have a better performance?

Thanks for any answer!

PS: I use HC in combo with DVD Rebuilder.

[Dark Shikari]

Start by raising AQ; you may also find a higher LUMGAIN useful.

[Fr4nz]

Start by raising AQ; you may also find a higher LUMGAIN useful.

Hi Dark.

I tried to follow your advice maxing AQ (4) and luma (4) parameters but nothing happens. The the "problem" I've described is present also in zones that have uniform coloring.

In order to explain better the problem take a look at these two pics, taken from the first chapter of the original "Milk" movie encoded with HC:

http://img10.imageshack.us/img10/8540/screen2rxn.th.png (http://img10.imageshack.us/my.php?image=screen2rxn.png)

http://img10.imageshack.us/img10/3759/screen1tlj.th.png (http://img10.imageshack.us/my.php?image=screen1tlj.png)

In the first screencap you can see some blocking at the right dark stripe area. In the original movie and in the CCE encoding (with same bitrate) these blocks doesn't appear.

In the second screencap you can see other small blocks at the middle, where there is a (omogeneously coloured) azure "bubble". Again, in the original movie and in the CCE encoding these phenomena doesn't show up.

The only "countermeasure" I've found effective against these "little blocks" was to raise the bias to [90-100], but this means that the encoded video will be almost CBR.

As a last note I have to say that these blocks appear and disappear during the playback.

Hope someone can help, because I found HC Enc a great-great piece of free software as the magnificent x264 :)

EDIT: These are screencaps from the original movie (useful for a comparison):

http://img14.imageshack.us/img14/8427/screen2dxt.th.png (http://img14.imageshack.us/my.php?image=screen2dxt.png)

http://img14.imageshack.us/img14/5667/screen1u.th.png (http://img14.imageshack.us/my.php?image=screen1u.png)

[Sharktooth]

try using a different custom matrix.

[Fr4nz]

try using a different custom matrix.

I tried DVD Rebuilder matrix, MPEG matrix and FOX matrices hardcoded in HC Enc...no improvements so far.

Do you have any suggestion?

Anyway, what suprises me is that the encoding is not "extreme" (very low bitrate/strange parameters/etc.), so for the HC encoder shouldn't be difficult to obtain results similar to CCE...

[Marius-the-Mad]

It will probably not change much, considering the bitrate alone helps, but... What happens if you up the DC precision to 10? Since it's a strange problem, as you say, maybe it's worth a try.

Regarding your first screenshot: I can see the blocks only with brightness and contrast set to maximum. That's on a 24 inch LCD with an 8-bit H-IPS screen - and that setting is way, way above the calibrated brightness and contrast settings, which in turn are already way too bright for normal, non-graphic work. Maybe the first problem is only visible on some particular types of screen? Just a wild guess.

[Fr4nz]

It will probably not change much, considering the bitrate alone helps, but... What happens if you up the DC precision to 10? Since it's a strange problem, as you say, maybe it's worth a try.

Regarding your first screenshot: I can see the blocks only with brightness and contrast set to maximum. That's on a 24 inch LCD with an 8-bit H-IPS screen - and that setting is way, way above the calibrated brightness and contrast settings, which in turn are already way too bright for normal, non-graphic work. Maybe the first problem is only visible on some particular types of screen? Just a wild guess.

Hello Marius!

I tried with DC 10, but there's no improvement.

Passing to the first screenshot considerations: yes, blocks in the dark right stripe are noticeable only if you set brightness and contrast with high values (but the CCE encoding doesn't introduce blocks there), but blocks in the second screenshot are noticeable also with "normal" settings and that is not good :(

What is intriguing me is: why CCE performs better than HC with settings that are not "extreme"? Maybe we could conduct some test with other movies in order to give some feedback to Hank.

[Marius-the-Mad]

Hmm...

The blocks on the first screenshot may be by design. They are invisible and you won't ever see them unless you cheat, so to speak. ;)

As for the second one - curious thing, indeed. And I just remembered something.
Not long ago, HC gave me blocks on "Akira". It's an anime, but the second screenshot's characteristic may be similar enough. Try to disable AQ completely - it cured the problem for me then.

HC's AQ usually helps tremendously - but not always. However, if that doesn't help, I'm out of ideas. Maybe someone more experienced will be able to help.

Good luck. :)

[Fr4nz]

Hmm...

The blocks on the first screenshot may be by design. They are invisible and you won't ever see them unless you cheat, so to speak. ;)

I swear I'm not cheating, if requested I'll provide the m2v encoded file.

As for the second one - curious thing, indeed. And I just remembered something.
Not long ago, HC gave me blocks on "Akira". It's an anime, but the second screenshot's characteristic may be similar enough. Try to disable AQ completely - it cured the problem for me then.

I've tried that but with no success. The only thing that worked was to raise "VBR bias" to 90-100 (but this means the video will be almost CBR).

[manolito]

The only thing that worked was to raise "VBR bias" to 90-100 (but this means the video will be almost CBR).
Interesting thread...
I suspect that HC uses a different algorithm for multipass bitrate distribution compared to CCE. Maybe you could try two more things:

1. Use a matrix which assigns more bitrate to flat areas and takes these bits away from complicated areas. I am a big fan of the two following matrices. They were posted some time ago by Manono (I believe he extracted them from commercial DVDs), and they both have relatively low coefficients to the upper left and high values to the lower right. At your average bitrate I would probably go with the "Sony Medium".

Matrix Manono Standard
*CUSTOMMATRIX
8 8 8 9 11 13 14 17
8 8 9 11 13 13 14 17
8 8 11 12 13 14 17 94
9 11 13 13 14 17 17 94
11 11 13 13 14 17 94 94
13 13 14 16 17 20 94 94
13 13 14 17 94 94 94 94
13 14 17 94 94 94 94 94

12 12 13 14 15 16 22 26
12 13 14 15 16 22 26 32
13 14 15 16 22 26 32 41
14 15 16 22 26 32 41 53
15 16 22 26 32 41 53 94
16 22 26 32 41 53 70 94
22 26 32 41 53 70 94 94
26 32 41 53 94 94 94 94


Matrix Sony Medium
*CUSTOMMATRIX
8 12 13 14 15 16 19 22
12 13 14 15 16 19 22 26
13 14 15 16 19 22 26 32
14 15 16 19 22 26 32 41
15 16 19 22 26 32 41 53
16 19 22 26 32 41 53 70
19 22 26 32 41 53 70 94
22 26 32 41 53 70 94 127

12 12 13 14 15 16 19 22
12 13 14 15 16 19 22 26
13 14 15 16 19 22 26 32
14 15 16 19 22 26 32 41
15 16 19 22 26 32 41 53
16 19 22 26 32 41 53 70
19 22 26 32 41 53 70 94
22 26 32 41 53 70 94 127


2. Try to encode in CQ_MAXBITRATE mode. If you enabled the HC log for your 2-pass encodes the log file tells you the average quantizer. Look what happens if you use this quantizer.


Cheers
manolito

[gizzin]

I use to find that hc was blocky in dark areas that are flat. But lumgain fixed that problem for me. And at the bitrate it really doesn't make much sense to me for it to have any artifacts at all. A simple solution would not to use HC, and use CCE. Maybe its actually a encoding bug in hc?

[Fr4nz]

Hi Manolito, here are the tests I've made, I hope they're useful:

1) Encoding with "Sony" matrix:

log:

-----------------------------------------
| HCenc - MPEG2 encoder - rel. 0.23.0.0 |
-----------------------------------------

MPEG profile@level: MP@ML
input: e:\work\mainmovie\milk_scn\video_ts\vts_01_1.d2v
output: E:\prova.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 0 - 5000
framerate: 25.000
aspect ratio: 16:9
bitrate Kb/s: 4500
max. bitrate Kb/s: 8000
pulldown: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: auto, TFF
goplen,B-pic: AUTO
dc_precision: 9
scan method: auto
bias: 20
chapter frames: 0
time code: 0 0 0 0
CPU: SSSE3
priority: idle
SMP active: yes
matrix: CUSTOM, adaptive
luminance gain: yes, level 2
adaptive quantization: yes, strength 2

------------------
| source stats |
------------------

nr. of frames in source: 184514
width*height: 720x576
fps: 25.000
nr. of frames to encode: 5001
frames to encode: 0 - 5000

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:00:42 (42.14 s)
fps: 118.7

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 4500 kb/s
estimated output file length: 109885 kB
intermediate encoding time: 0.00 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 0:00:41 (40.84 s)
fps: 122.4

------------------
| encoding stats |
------------------


adapted intra matrix used
adapted non-intra matrix used

nr. of gops: 351
nr. of frames: 5001
nr. of I-frames: 351
nr. of P-frames: 1390
nr. of B-frames: 3260
average bitrate: 4499
minimum bitrate: 149
maximum bitrate: 7857

bytes in bitstream: 112562532
bits in bitstream: 900500256

average Quantizer: 6.268

total CPU time: 0:02:36 (156.02 s)
total elapsed time: 0:01:23 (83.08 s)


pics:

http://img22.imageshack.us/img22/3749/screen1ljj.th.png (http://img22.imageshack.us/my.php?image=screen1ljj.png)

http://img22.imageshack.us/img22/4984/screen2zhj.th.png (http://img22.imageshack.us/my.php?image=screen2zhj.png)

As you can see there's no improvement in the "dark stripe" scene; a thing that I didn't mention before is that if you watch the encoding carefully you'll see that these blocks "flashes": in other words they appear and disappear; maybe this could be a useful clue?

In the "bubble" scene there's an improvement, blocks are less and less visible;

2) Encoding with "manono" matrix:

log:

-----------------------------------------
| HCenc - MPEG2 encoder - rel. 0.23.0.0 |
-----------------------------------------

MPEG profile@level: MP@ML
input: e:\work\mainmovie\milk_scn\video_ts\vts_01_1.d2v
output: E:\prova2.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 0 - 5000
framerate: 25.000
aspect ratio: 16:9
bitrate Kb/s: 4500
max. bitrate Kb/s: 8000
pulldown: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: auto, TFF
goplen,B-pic: AUTO
dc_precision: 9
scan method: auto
bias: 20
chapter frames: 0
time code: 0 0 0 0
CPU: SSSE3
priority: idle
SMP active: yes
matrix: CUSTOM, adaptive
luminance gain: yes, level 2
adaptive quantization: yes, strength 2

------------------
| source stats |
------------------

nr. of frames in source: 184514
width*height: 720x576
fps: 25.000
nr. of frames to encode: 5001
frames to encode: 0 - 5000

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:00:42 (42.12 s)
fps: 118.7

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 4500 kb/s
estimated output file length: 109885 kB
intermediate encoding time: 0.00 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 0:00:41 (40.83 s)
fps: 122.5

------------------
| encoding stats |
------------------


adapted intra matrix used
adapted non-intra matrix used

nr. of gops: 351
nr. of frames: 5001
nr. of I-frames: 351
nr. of P-frames: 1390
nr. of B-frames: 3260
average bitrate: 4499
minimum bitrate: 149
maximum bitrate: 7768

bytes in bitstream: 112559553
bits in bitstream: 900476424

average Quantizer: 6.421

total CPU time: 0:02:36 (155.55 s)
total elapsed time: 0:01:23 (83.06 s)


pics:

http://img7.imageshack.us/img7/5279/screen2amo.th.png (http://img7.imageshack.us/my.php?image=screen2amo.png)

http://img16.imageshack.us/img16/8922/screen1t.th.png (http://img16.imageshack.us/my.php?image=screen1t.png)

With "manono" matrix I didn't notice significant improvements.

3) Costant quantizer encoding (CQ set at "5"):

log:

-----------------------------------------
| HCenc - MPEG2 encoder - rel. 0.23.0.0 |
-----------------------------------------

MPEG profile@level: MP@ML
input: e:\work\mainmovie\milk_scn\video_ts\vts_01_1.d2v
output: E:\prova3.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 0 - 5000
framerate: 25.000
aspect ratio: 16:9
bitrate: NA
max. bitrate Kb/s: 8000
pass: 1 (Constant Quant)
constant Q: 5.000
pulldown: no
closed gops: no
VBV check: yes
scene change det.: yes
interlaced: auto, TFF
goplen,B-pic: AUTO
dc_precision: 9
scan method: auto
bias: 0
chapter frames: 0
time code: 0 0 0 0
CPU: SSSE3
priority: idle
SMP active: yes
matrix: MPEG, adaptive
luminance gain: yes, level 2
adaptive quantization: yes, strength 2

------------------
| source stats |
------------------

nr. of frames in source: 184514
width*height: 720x576
fps: 25.000
nr. of frames to encode: 5001
frames to encode: 0 - 5000

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:00:48 (47.75 s)
fps: 104.7

------------------
| encoding stats |
------------------


adapted intra matrix used
adapted non-intra matrix used

nr. of gops: 351
nr. of frames: 5001
nr. of I-frames: 351
nr. of P-frames: 1390
nr. of B-frames: 3260
average bitrate: 4319
minimum bitrate: 149
maximum bitrate: 7888

bytes in bitstream: 108039402
bits in bitstream: 864315216

average Quantizer: 5.001

total CPU time: 0:01:25 (47.75 s)
total elapsed time: 0:00:48 (47.87 s)


pics:

http://img508.imageshack.us/img508/5224/screen2a.th.png (http://img508.imageshack.us/my.php?image=screen2a.png)

http://img17.imageshack.us/img17/2759/screen1hht.th.png (http://img17.imageshack.us/my.php?image=screen1hht.png)

Again, no improvements. But having used the CQ-mode this fact really suprises me.

PS: IMHO it would be very useful for all to see "with your own eyes" the results of HC encodings in order to understand better these artifacts. So, If you want, I could upload the original segment of the Milk movie I'm using for these tests: it's composed of the first 5000 frames of the movie and its size is 170 Mbytes.
Just tell me if it's permitted to post the link here and where to upload it :)

[Sharc]

Unless already done, you should crop the black borders first - and possibly resize the active picture - to comply with mod16 rule.
Then add borders which do also comply to mod 16.
Full mod16 compliancy for the active picture and the borders ensures that HCenc does not waste any bits for encoding the sharp transition between the active picture and the black borders.

Once you know in which frames the blocks appear, you may use the zone function of HCenc and raise the bitrate manually for these critical frames.

I doubt a little if these blocks are disturbing when watching the movie. Maybe yo find CCE to perform better in this respect, however you may discover other scenes where CCE produces mosquito noise while HCenc is clean....

[Fr4nz]

Unless already done, you should crop the black borders first - and possibly resize the active picture - to comply with mod16 rule.
Then add borders which do also comply to mod 16.
Full mod16 compliancy for the active picture and the borders ensures that HCenc does not waste any bits for encoding the sharp transition between the active picture and the black borders.

Once you know in which frames the blocks appear, you may use the zone function of HCenc and raise the bitrate manually for these critical frames.

I doubt a little if these blocks are disturbing when watching the movie. Maybe yo find CCE to perform better in this respect, however you may discover other scenes where CCE produces mosquito noise while HCenc is clean....

Excuse me Sharc, but encoding directly at the original 720x576 res should be mod16 compliant, isn't it?

[Sharc]

Excuse me Sharc, but encoding directly at the original 720x576 res should be mod16 compliant, isn't it?
No, unfortunately not - in my experience. I have seen many DVDs violating the mod16, i.e. the active picture is not mod16 and just "centered" in the 720x576 area, means when re-encoding the transition between active picture an border falls right somewhere within a macroblock.

[Fr4nz]

No, unfortunately not - in my experience. I have seen many DVDs violating the mod16, i.e. the active picture is not mod16 and just "centered" in the 720x576 area, means when re-encoding the transition between active picture an border falls right somewhere within a macroblock.

Ok Sharc, thanks for the clarification.

Please, could you provide me an example of an avs script which does the things you've suggested (http://forum.doom9.org/showpost.php?p=1278348&postcount=13) me to do?

Thanks :)

[Sharc]

Here an example assuming top and bottom borders of 72 pixels each (not mod16!) and an active picture of 720x432

#mod16 PAL adjustment
crop(0,72,-0,-72)
AddBorders(0,64,0,80)

Now the active picture and the borders are all mod16 compliant; the active picture is slightly shifted upwards.

Or another one - worse even - where the borders are 76 pixels top/bottom each (not mod16) and the active picture is 720x424 (not mod16):

#mod16 PAL adjustment
crop(0,76,-0,-76)
Spline36Resize(720,432)
AddBorders(0,64,0,80)

[Fr4nz]

Here an example assuming top and bottom borders of 72 pixels each (not mod16!) and an active picture of 720x432

#mod16 PAL adjustment
crop(0,72,-0,-72)
AddBorders(0,64,0,80)

Now the active picture and the borders are all mod16 compliant; the active picture is slightly shifted upwards.

Or another one - worse even - where the borders are 76 pixels top/bottom each (not mod16) and the active picture is 720x424 (not mod16):

#mod16 PAL adjustment
crop(0,76,-0,-76)
Spline36Resize(720,432)
AddBorders(0,64,0,80)

Ok Sharc, I used your first script but artifacts are still present...it didn't work :\

[Sharc]

Even the original bubble picture shows compression artefacts (blocks) - just weaker. HCenc cannot remove these by compressing these frames even more.
Still, if you want to reduce the artefacts you may use the zone option for these critical frames and boost the bitrate. One has however to keep in mind that the extra bits will be stolen from other scenes - assuming a fixed total file size.
(B.t.w. I normally use a bias of 30 with HCenc - but some weak blocking may still be observed in fading or flat scenes, or exceptionally after a scene change for the duration of a few frames.)

[Fr4nz]

Even the original bubble picture shows compression artefacts (blocks) - just weaker. HCenc cannot remove these by compressing these frames even more.

Well, you really can notice the difference (while with CCE not).

Still, if you want to reduce the artefacts you may use the zone option for these critical frames and boost the bitrate. One has however to keep in mind that the extra bits will be stolen from other scenes - assuming a fixed total file size.
(B.t.w. I normally use a bias of 30 with HCenc - but some weak blocking may still be observed in fading or flat scenes, or exceptionally after a scene change for the duration of a few frames.)

I'll keep your considerations in mind for the future. :)

[Alex-Kid]

It hasn't been mentioned, but maybe the "Restart Avisynth on 2nd pass" can help a little. I required it (frequently) in some codings, and used to produce some blockiness when off.

[Fr4nz]

It hasn't been mentioned, but maybe the "Restart Avisynth on 2nd pass" can help a little. I required it (frequently) in some codings, and used to produce some blockiness when off.

Just tried but this unfortunately didn't change anything :\

[Marius-the-Mad]

I swear I'm not cheating, if requested I'll provide the m2v encoded file.

Hi Fr4nz. I'm sorry if it sounded like an accusation. :( By "cheating" I meant: you can't see the blocks in the first screenshot unless you increase brightness and contrast to completely impractical levels. :) When you watch the movie normally, you do not perceive said blocks (as I understand), so, IMO, it's perfectly acceptable to have them there.

Good luck with your encodings. :) This is an interesting thread.

[manolito]

@Fr4nz
You said that CCE SP does not show the blockiness, so I am interested in your CCE settings.

Under the "Picture Quality" tab you can choose from a couple of filters, and some of them are enabled by default. For a comparison with HC all CCE filters must of course be unchecked.

I am also curious about your values for Bias and "Quantizer Characteristics".

Cheers
manolito

[gizzin]

HC inherently had problems with fade ins, outs. And dark black areas. So maybe its not entirely fixed? I still do occasionally see blockiness in fades.

[Fr4nz]

@Fr4nz
You said that CCE SP does not show the blockiness, so I am interested in your CCE settings.

Under the "Picture Quality" tab you can choose from a couple of filters, and some of them are enabled by default. For a comparison with HC all CCE filters must of course be unchecked.

I am also curious about your values for Bias and "Quantizer Characteristics".

Cheers
manolito

Hi Manolito, I use CCE-SP thru DVD Rebuilder. The settings I use for it are:

- VBR Bias: 20 (it can go down to 15, depends on the length of the movie)
- Qual Prec: 15
- VBR Passes: 2

That's all. I don't know if DVD-RB passes other settings to CCE-SP (probably yes, but I don't know which).

About filters: usually I don't use filters in DVD-RB, so CCE and HC are in the same "starting conditions" :)

[Emulgator]

Here comes a small trick.

You may tickle the encoder into spending more bitrate
where he thought he could save on bitrate by a simple script line.
In the avisynth input script type the following line Addgrain(6,0,0)
(you may try with less).

This helps with gradients, dark parts, fades, but adds grain and wastes some bitrate.

Find out what suits most with HC.

I found the same with HC, a bit too blocky in dark parts,
and even pushing Lumgain up to 4 did not help too much...

[Sharc]

You could also try to enable/disable redistribution and see if it makes a difference. It will change the bitrate allocation.

[Graigddu]

I have also tried to remove blockiness in dark areas in encodes with HC Enc
i found these to be "smoothed" to a certain extent by CCE even originals with some blockiness were also worked on,
as a workaround to this i tried different settings in HC Enc with no luck and then moved on to adding
avisynth filters in to a filterchain with some effect but the end result was a lot longer encode time
and mod16 errors quite often because of the filters used,
I moved from CCE originally as i had an older version which was slow in encoding but i have not been able to
increase the HCEnc results without a price,either speed or quality lost

[kehchen]

You should try AQ in version 0.24 (0.23 not support yet)
If encode with high bitrate, please try *INTRAVLC 1 as well.

[Fr4nz]

You should try AQ in version 0.24 (0.23 not support yet)
If encode with high bitrate, please try *INTRAVLC 1 as well.

Are you saying that in 0.23 AQ is not working??

[kehchen]

Hmm.. maybe I made the mistake, AQ should be working in 0.23
But I found I get better result in 0.24 especially in dark scene. Maybe you should try it out.

[Midzuki]

I think it wouldn't hurt to try a matrix that compresses less than the ones from FOX. For example:


08 08 08 09 09 10 10 11
08 08 09 09 10 10 11 11
08 09 09 10 11 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 12 13
10 10 11 11 12 12 13 14
10 11 11 12 12 13 14 15
11 11 12 12 13 14 15 16

08 08 08 08 08 08 08 08
08 08 08 08 08 08 08 08
08 08 08 08 08 08 08 08
08 08 08 08 08 08 08 08
08 08 08 08 08 08 08 08
08 08 08 08 08 08 08 08
08 08 08 08 08 08 08 08
08 08 08 08 08 08 08 08


HTH.