Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 ASP

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd February 2008, 00:30   #1  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
VAQ: Making Xvid's Adaptive Quantization Not Suck

I was bored, so I spent 15 minutes and wrote this patch to port x264's new VAQ to Xvid.

I haven't tested it much (only on this video), but it seems to work surprisingly well, despite MPEG-4 ASP's drastic limitations on quantizer deltas.

Patch [updated with minor fixes]

Xvidcore.dll [updated with latest patch]

To use it with Virtualdub or any VfW interface, just overwrite the XvidCore.dll in your /Windows/System32 or similar with the one I have provided.

To activate VAQ, just check off the Adaptive Quantization box; it replaces the normal luminance masking.

Note that I could not get it working correctly with mencoder; for some reason libavcodec seems to have its own lumimasking library that's overriding my patch. It works with xvid_encraw and VfW though...

Have fun!

Last edited by Dark Shikari; 23rd February 2008 at 01:37.
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 01:29   #2  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
If people want, I can add it to my xvid_encraw as a new option. Unfortunately I have no time to do my own tests at the moment.
squid_80 is offline   Reply With Quote
Old 22nd February 2008, 02:03   #3  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by squid_80 View Post
If people want, I can add it to my xvid_encraw as a new option. Unfortunately I have no time to do my own tests at the moment.
In the patch's current form it overwrites lumimasking, so one would need a separate version (it uses the same commandline as lumimasking).
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 02:23   #4  |  Link
Zep
Registered User
 
Join Date: Jul 2002
Posts: 587
Quote:
Originally Posted by Dark Shikari View Post
I was bored, so I spent 15 minutes and wrote this patch to port x264's new VAQ to Xvid.

I haven't tested it much (only on this video), but it seems to work surprisingly well, despite MPEG-4 ASP's drastic limitations on quantizer deltas.
can you link to the without patch encode for comparison?
Zep is offline   Reply With Quote
Old 22nd February 2008, 03:26   #5  |  Link
elguaxo
Registered User
 
elguaxo's Avatar
 
Join Date: Jun 2006
Posts: 260
elguaxo is offline   Reply With Quote
Old 22nd February 2008, 03:29   #6  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by Zep View Post
can you link to the without patch encode for comparison?
That is the x264 version of the video

Do Xvid test encodes yourself.
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 03:33   #7  |  Link
Ranguvar
Registered User
 
Ranguvar's Avatar
 
Join Date: Feb 2007
Location: ::1
Posts: 1,236
Awesome!

But I SO wish you had done this before I backed up a bunch of stuff to Xvid

Squid_80, please do. I will test if I get any time, which is most likely as I have this week off.

Dark Shikari, can this Xvidcore be used against any version of the Xvid codec? I have a recent build of 1.2.-127.
Ranguvar is offline   Reply With Quote
Old 22nd February 2008, 03:37   #8  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by Ranguvar View Post
Awesome!

But I SO wish you had done this before I backed up a bunch of stuff to Xvid

Squid_80, please do. I will test if I get any time, which is most likely as I have this week off.

Dark Shikari, can this Xvidcore be used against any version of the Xvid codec? I have a recent build of 1.2.-127.
This is the latest CVS snapshot of Xvid + my patch.
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 08:10   #9  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
Quote:
Originally Posted by Dark Shikari View Post
In the patch's current form it overwrites lumimasking, so one would need a separate version (it uses the same commandline as lumimasking).
The beauty of the xvid's plugin system - I can add however many plugins I want to encraw and xvidcore.dll doesn't need to be updated.
squid_80 is offline   Reply With Quote
Old 22nd February 2008, 09:46   #10  |  Link
G_M_C
Registered User
 
Join Date: Feb 2006
Posts: 1,076
Quote:
Originally Posted by Dark Shikari View Post
I was bored, so I spent 15 minutes and wrote this patch to port x264's new VAQ to Xvid[...]
OMG

I'd allmost wish you'd be bored more often.

.... But on the other hand ...

If you do things like these VAQ-patches for X264 & Xvid while bored ... i'd hate to think of things you could do while really interested



Anyway,just a question; Could this same patch / idea be ported to Hank's HCEnc MPEG2 encoder ? I use that one quite often to make (author) DVD's.

I use Hanks luminance-adative code, wich works surprisingly well. But that code works only on "darker" scenes, not on flat(ter) scenes like the VAQ does. I would be really interested in such a feature

Last edited by G_M_C; 22nd February 2008 at 09:51.
G_M_C is offline   Reply With Quote
Old 22nd February 2008, 10:55   #11  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by G_M_C View Post
Anyway,just a question; Could this same patch / idea be ported to Hank's HCEnc MPEG2 encoder ? I use that one quite often to make (author) DVD's.
I don't see why not...

All I need is a link to the source.
Quote:
Originally Posted by G_M_C View Post
If you do things like these VAQ-patches for X264 & Xvid while bored ... i'd hate to think of things you could do while really interested
Well x264's took quite a while--but once I had it working I can easily port it to any other similar encoder with very little work.
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 16:35   #12  |  Link
hank315
HCenc author
 
Join Date: Nov 2003
Location: Netherlands
Posts: 570
I'm already working on it for some time, the general idea is about the same, variance based.
Will be in the next HCenc release.

Just had a look at the implementation in Xvid, well, I'm a C n00b but when I do something like this in Fortran it will overflow on high luminance values:

Code:
  for (k = 0; k < 16; k++)
      for (l = 0; l < 16; l++)
         {
             int val = ptr[k*data->current.stride[0] + l];
             sum += val;
             sum_of_squares += val * val;
         }

  /* Variance = SSD - SAD^2 / (numpixels) */
  int variance = sum_of_squares - sum * sum / 256;
__________________
HCenc at: http://hank315.nl
hank315 is offline   Reply With Quote
Old 22nd February 2008, 17:09   #13  |  Link
elguaxo
Registered User
 
elguaxo's Avatar
 
Join Date: Jun 2006
Posts: 260
Quote:
Originally Posted by Dark Shikari View Post
Edit: I accidentally left a debug statement in there, so if you're using xvid_encraw... well... you might notice "THISISATEST" constantly printing
I'm using xvid_encraw. I have no idea how to compile this myself. Could you or somone else post a binary without the debug text? Thanks in advance!
elguaxo is offline   Reply With Quote
Old 22nd February 2008, 17:37   #14  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
I think that patch isn't really proper (at least it didn't work for me).
After some slight editing of it though I managed to get it to work (with a fuzz with the last hunk, but I checked the luminance_masking.c and everything was patched correctly)

Now off to test. I think the patch doesn't have that debug line Dark was talking about so I'll post a build in a bit I guess and see if that helps (it's only a debug line though so does it really matter?)

Edit:
Build:
http://www.fileducky.com/qRILYtae/

It's built with pthreads enabled.

Last edited by TheRyuu; 22nd February 2008 at 17:48.
TheRyuu is offline   Reply With Quote
Old 22nd February 2008, 17:58   #15  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by wizboy11 View Post
I think that patch isn't really proper (at least it didn't work for me).
After some slight editing of it though I managed to get it to work (with a fuzz with the last hunk, but I checked the luminance_masking.c and everything was patched correctly)

Now off to test. I think the patch doesn't have that debug line Dark was talking about so I'll post a build in a bit I guess and see if that helps (it's only a debug line though so does it really matter?)
Yes, I removed the debug line in the patch so that people didn't accidentally include it Only my build has it.
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 18:00   #16  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by hank315 View Post
I'm already working on it for some time, the general idea is about the same, variance based.
Will be in the next HCenc release.

Just had a look at the implementation in Xvid, well, I'm a C n00b but when I do something like this in Fortran it will overflow on high luminance values:

Code:
  for (k = 0; k < 16; k++)
      for (l = 0; l < 16; l++)
         {
             int val = ptr[k*data->current.stride[0] + l];
             sum += val;
             sum_of_squares += val * val;
         }

  /* Variance = SSD - SAD^2 / (numpixels) */
  int variance = sum_of_squares - sum * sum / 256;
The maximum value that sum_of_squares can reach is (255 * 255 * 256), or a bit under 2^24, so it is not a problem.

The maximum value that sum can reach is 255*256... so yes, you are right; sum*sum could overflow a signed integer.

The correction would be to make sum and ssd unsigned integers.
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 18:54   #17  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
If the product is going to be divided by 256, why not bitshift before the multiply?
squid_80 is offline   Reply With Quote
Old 22nd February 2008, 18:58   #18  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by squid_80 View Post
If the product is going to be divided by 256, why not bitshift before the multiply?
Rounding error. Same problem occurred in x264; that was the fix between 0.46 and 0.47.
Dark Shikari is offline   Reply With Quote
Old 22nd February 2008, 19:07   #19  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
Divide by 256, with rounding:
(x+128)>>8

No good?

Last edited by squid_80; 22nd February 2008 at 19:12. Reason: oops, that's just plain rounding. Rounding upwards would be +251.
squid_80 is offline   Reply With Quote
Old 22nd February 2008, 19:10   #20  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by squid_80 View Post
Divide by 256, rounding upwards:
(x+128)>>8

No good?
It won't give the same result as multiplying before shifting, IIRC.

Patch updated with unsigned fix.

Last edited by Dark Shikari; 22nd February 2008 at 19:12.
Dark Shikari is offline   Reply With Quote
Reply

Tags
xvid aq, xvid vaq

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:49.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.