Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th March 2008, 06:32   #1  |  Link
plugh
A hollow voice says
 
Join Date: Sep 2006
Posts: 269
xvid's chroma optimizer

Is there a way to impliment this in a script so I can 'see' what it does?
Code:
image_chroma_optimize(IMAGE * img, int width, int height, int edged_width)
{
	int x,y;

	for (y = 1; y < height/2 - 1; y++)
	for (x = 1; x < width/2 - 1; x++)
	{
#define IS_PURE(a)  ((a)<=16||(a)>=235)
#define IMG_Y(Y,X)	img->y[(Y)*edged_width + (X)]
#define IMG_U(Y,X)	img->u[(Y)*edged_width/2 + (X)]
#define IMG_V(Y,X)	img->v[(Y)*edged_width/2 + (X)]

		if (IS_PURE(IMG_Y(y*2  ,x*2  )) &&
			IS_PURE(IMG_Y(y*2  ,x*2+1)) &&
			IS_PURE(IMG_Y(y*2+1,x*2  )) &&
			IS_PURE(IMG_Y(y*2+1,x*2+1)))
		{
			IMG_U(y,x) = (IMG_U(y,x-1) + IMG_U(y-1, x) + IMG_U(y, x+1) + IMG_U(y+1, x)) / 4;
			IMG_V(y,x) = (IMG_V(y,x-1) + IMG_V(y-1, x) + IMG_V(y, x+1) + IMG_V(y+1, x)) / 4;
		}
	}
('edged_width' is width rounded up to mod 16 plus padding...)

It doesn't have to be efficient, as I don't intend to actually run a full movie through such a script. I just want to compare before and after on some frames...
plugh is offline   Reply With Quote
Old 27th March 2008, 08:35   #2  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
No problem to do that with MaskTools. Basic operation is just this:

Code:
mt_luts(last,last,mode="average",pixels="0 -1 -1 0 0 1 1 0",expr="y",Y=2,U=3,V=3)
To complete cloning of Xvid's operation, one would additionally need to build a mask for Y values Y<16 + Y>235, expand it with mt_expand(mode="both"), and use that for masking. Not sure if that's really needed for your tests, though.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 27th March 2008, 20:47   #3  |  Link
plugh
A hollow voice says
 
Join Date: Sep 2006
Posts: 269
Thanks for the reply. Is that - hmmm what term to use - 'self referential' like the xvid code snippet?

The calc for the (I)th pixel uses pixel (I+1). When it then goes to calc the (I+1)th pixel, it uses the value it just calculated - ie the (I-1)th pixel.

Ditto for the vertical direction - when calc'ing pixel in row J, it uses pixel in row (J+1). When it gets around to calc'ing that pixel in row (J+1) it uses the value in the previous row that it calc'ed from this pixel.

So all references to the left and up use previously calc'ed pixel values, those to the right and down use actual pixel values. That looks fishy to me, which is why I want to visualize the results...

Last edited by plugh; 27th March 2008 at 20:49.
plugh is offline   Reply With Quote
Old 27th March 2008, 23:03   #4  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Oops, my bad. Too many years of not using a programming language ... now that you mention it, I see that loopback effect.

But now, I don't see a (sane) way to do that with an Avisynth script, except you really want to run a chain of 360 times (mt_luts + mt_merge) for a 720*y frame. It should be easier to make a plugin that does the filtering in such a back-injecting way.

When looking at the resulting weights, my guess is that the effect isn't too bad. 50% of the samples used for filtering are already filtered. Those filtered samples contain 25% weight of the actual pixel ... no hard science, but since chroma filtering is quite forgiving anyway, probably it just works out. But nonetheless, a slight color shift towards upper-left is to be expected, roughly in the range of 1/4 chroma pel.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 27th March 2008 at 23:05.
Didée is offline   Reply With Quote
Old 28th March 2008, 01:48   #5  |  Link
plugh
A hollow voice says
 
Join Date: Sep 2006
Posts: 269
ok, that's what I thought. I've never looked at the avisynth sdk, so don't know how involved it might be...

Glad you concur that it seems a bit odd. I never use the 'Chroma optimizer' but happenned to look at the code the other day and said WTF? Hence my desire to visualize the result...
plugh is offline   Reply With Quote
Old 28th March 2008, 06:29   #6  |  Link
sixide
Registered User
 
Join Date: Jun 2005
Posts: 9
I did a single frame test in MATLAB. I found a JPEG screenshot of No Country For Old Men with Google.

I am assuming that images are indexed the same way, from top left to bottom right.

I used a bilinear resize for upsizing/downsizing the chroma planes, and the rgb2ycbcr/ycbcr2rgb functions for conversions. I know that Cb/Cr are not U/V, but the effect on those planes will be the same. The effect on the final color is probably not.

Original:
http://img186.imageshack.us/my.php?i...roldmeniq6.jpg
Optimized:
http://img149.imageshack.us/my.php?i...cfomopthh1.png
Scaled Cb difference:
http://img183.imageshack.us/my.php?image=ncdiffuhg3.png
Scaled Cr difference:
http://img338.imageshack.us/my.php?image=ncdiffvws8.png
Original Cb:
http://img339.imageshack.us/my.php?image=ncumw5.png
Original Cr:
http://img214.imageshack.us/my.php?image=ncvwn5.png
Optimized Cb:
http://img214.imageshack.us/my.php?image=ncoull2.png
Optimized Cr:
http://img297.imageshack.us/my.php?image=ncovbe0.png

The max diff for luma was +/-0, Cb was +9/-39, Cr was +5/-8.

As to why this is "optimized"? I don't know, I'm curious. Isn't this pretty much a chroma smoothing operation?
sixide is offline   Reply With Quote
Old 28th March 2008, 06:39   #7  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
I would assume it makes the chroma easier to encode for Xvid, maybe.
Dark Shikari is offline   Reply With Quote
Old 28th March 2008, 14:33   #8  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 221
What the algorithm does (note that x,y range as [1..width/2) and [1..height/2) respectively) is (in pseudo-code):

Code:
ForEach interior chroma pixel (ie for all interior 2x2 luma pixel blocks)
    If all luma pixels in the 2x2 pixel block are pure black or white Then
        chroma pixel value = Average of adjacent (ie +/-1) left,top,right,bottom chroma pixels
I havent studied xvid's code so I can only guess that the intention of the code is to balance somehow the chroma of image areas that exhibit limiting luma values.
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 29th March 2008, 16:01   #9  |  Link
plugh
A hollow voice says
 
Join Date: Sep 2006
Posts: 269
Quote:
Originally Posted by sixide
I did a single frame test in MATLAB. [...]Isn't this pretty much a chroma smoothing operation?
wow, thanks!... Looking at the images, nothing jumps out at me, though.
Quote:
Originally Posted by gzarkadas View Post
Code:
ForEach interior chroma pixel (ie for all interior 2x2 luma pixel blocks)
    If all luma pixels in the 2x2 pixel block are pure black or white Then
        chroma pixel value = Average of adjacent (ie +/-1) left,top,right,bottom chroma pixels
What you describe would be the case if the result of each pixel calc was stored in a seperate (ie a new) image array. But since each calc'd pixel is stored back into the original image, pixels down and to right make use of modified pixels to left and up (which were calc'd from virgin pixels to right and down).

edit: example - take for example chroma at [x=2,y=2], here's where portions of its value gets spread to...
Code:
	1	2	3	4	5

1	0	1/4	1/16	1/64	1/256	
2	1/4	1/8	3/64	1/64	5/1024	
3	1/16	3/64	3/128	5/512	
4	1/64	1/64	5/512	5/1024	
5	1/256	5/1024

Last edited by plugh; 29th March 2008 at 16:59.
plugh is offline   Reply With Quote
Old 29th March 2008, 22:32   #10  |  Link
sixide
Registered User
 
Join Date: Jun 2005
Posts: 9
It seems to me that it should work on a copy of the original frame, too.

It is probable that not many frames have huge areas of luma outside (16,135), so the propagation may be minimal.
sixide is offline   Reply With Quote
Old 29th March 2008, 23:32   #11  |  Link
plugh
A hollow voice says
 
Join Date: Sep 2006
Posts: 269
I agree, and also considering the magnitudes involved (ie 8 bit) propogation is probably at most three cells. But if the intent, for example, was for [2,2] to use the original [2,1] and [1,2], they've (potentially) been clobberred by the time it gets to [2,2].

Anyway, Thanks all...
plugh is offline   Reply With Quote
Old 30th March 2008, 01:52   #12  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 221
Well, no need to guess anymore. After reviewing my understanding of the chroma optimisation algorithm thanks to plugh's correction, I gave up my attempts to create a script-based solution and made a plugin for this.

The plugin is attached in the message; it contains the source, a debug-dll and three test-case avs scripts. It is a fresh alpha, so please be kind with me ; any comments are welcomed of course.
Attached Files
File Type: zip chroma_optimize v0.1.0 (alpha).zip (28.6 KB, 111 views)
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 31st March 2008, 04:39   #13  |  Link
plugh
A hollow voice says
 
Join Date: Sep 2006
Posts: 269
Wow, thanks. I'll play with it a bit, but at this point it seems pretty certain that the xvid code is borked - I sent an message to the mailing list...
plugh is offline   Reply With Quote
Old 2nd April 2008, 20:59   #14  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 221
I agree with you. I downloaded the xvid source (1.1.3) and from the comments just above the function body it seems to me to be the case that the intention was to produce a (non-recursive) closest -neighbourhood average but the fact of the in-place substitution was not taken in account.

I have made an updated version of the plugin that supports both modes of averaging, recursive (like xvid's current) and non-recursive (as I pressume it should be). It also fixes some bugs discovered. Attached.

From the tests I have made so far the second, non-recursive, mode produces a more calm effect located only at the frontier of black/white regions with other colors. I upload an image as example, produced by the following script:

Code:
LoadPlugin("chroma_optimize.dll")
LoadPackage("avslib", "array")
LoadModule("avslib", "filters", "stack")

# test clip generation

bottom = BlankClip(width=320, height=240, pixel_type="YV12", color=color_orange)
top_w = bottom.BlankClip(width=60, height=60, color=color_white)
top_b = bottom.BlankClip(width=60, height=60, color=color_black)

Overlay(bottom, top_w, x=60, y=60)
Overlay(top_w, x=180, y=30)
Overlay(top_b, x=180, y=140)
Overlay(top_b, x=130, y=80)
Overlay(bottom.BilinearResize(120, 40), x=132, y=90)

# filter, diff, scale diffs, add borders & stack

src = ArrayCreate(last, last)
dst = ArrayOpArrayFunc(src, "true,false", "ChromaOptimize") # [ old (xvid) , new ] style
diff = ArrayOpArrayFunc(dst, src, "Subtract").ArrayOpFunc("Levels", "124,1,132,0,255")
Stack(ArrayPlex(src, dst, diff).ArrayOpFunc("AddBorders", "2,2,2,2"), 2, 3)

# output frame

ConvertToRGB24()
ImageWriter("comp", 1, 1, "png", false)
It also results in slightly more compressible encoded videos compared to unfiltered ones at least when encoding to MPEG4 (with xvid) and also to MOV (with ffmpeg); this was not observed when encoding to MPEG2 or WMV (with ffmpeg). However my test examples are very few and thus these results should be considered as preliminary only and possibly subject to future revising.

Thus a possible use of the plugin is as a final step in processing to squize a few more bytes from the encoded clip's size if the target format allows it. And of course as a replacement to the xvid's "optimize chroma" option for all those that agree with the view that xvid's implementation is not correct.
Attached Images
 
Attached Files
File Type: zip chroma_optimize v0.2.0 (alpha).zip (33.8 KB, 252 views)
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 5th April 2008, 03:35   #15  |  Link
plugh
A hollow voice says
 
Join Date: Sep 2006
Posts: 269
That image really drives it home!

I'm amazed that it seems the ENTIRE 'is pure' region is affected by the current algorithm.

Thanks again!
plugh is offline   Reply With Quote
Old 4th July 2008, 05:17   #16  |  Link
henryho_hk
Registered User
 
Join Date: Mar 2004
Posts: 889
Since we have interlaced encoding for XviD, will this plugin have an interlace mode too? ^_^
henryho_hk is offline   Reply With Quote
Old 26th December 2010, 11:10   #17  |  Link
henryho_hk
Registered User
 
Join Date: Mar 2004
Posts: 889
And... will it be ported back to XviD?

Edit: If we cache the original values of the last row and the "left" previous chroma pixels, there seems no need to work on an whole new frame buffer.

Last edited by henryho_hk; 3rd June 2011 at 02:08.
henryho_hk is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.