VHQ for beframes,...

[Selur]

Since the newest build over at gamr's site offers a enable VHQ for b-frames options and I didn't see a thread about it I'd like to start this thread to ask:

1. 'What to expect?'
(speed and quality wise; seems to be faster than 'old' VHQ4 without b-frames, or am I mistaken? haven't encoded a longer clip in a while so I'm not to sure ;) )

2. what are your experience with VHQ for b-frames ?

Cu Selur

[Koepi]

It's very new and has to be tweaked and tested. So it's not the time to have a thread like this, let sysKin develop it until it's ready please!

(for now it's mode decision for bframes and gives back 1.0 quality for 1.1 binary. )

[sysKin]

Well, I commited it to share it with the world, so I don't mind you testing or talking about it. Quite contrary in fact, I'd keep it to myself if that was what I wanted ;D

Anyway, VHQ for b-frames seems to work but something else in b-frames is broken. As a result, you actually need this VHQ to bring back the quality to the 1.0.1's level (more or less).

I'll try to find out what it is soon.

Radek

[xixi2000]

this a great job I think ,I want to test,downlaod from where?

[*.mp4 guy]

I would also like to see how the new bframe code behaves, and i seem to have forgotton the url to gam3r's sight.

[buzzqw]

http://xvid.gamrdev.com/

BHH

[xixi2000]

The test result is rather encouraging
The Prince And Me PAL 106min 576×320 128k MP3
there is a great improvenment on the picture quality.

[chilledoutuk]

i have also noticed a improvement in both the reduction of compresion artefacts and a smother more fluid motion estimation.

syskin dont suppose you have found whats causing the bframe problem yet then the quality will be amazing:D

[Andrey]

It would be great if sysKin posts it here, when he thinks he is done with b-frames. It would be interesting to test new code...

[virus]

I was ready to post a full PSNR curve comparison between the 1.0.1 and today's CVS but I see it's currently quite useless. The new code is apparently 0.03-0.06 dB below the old one :(

I said "apparently" 'cause I also have some problems patching the CVS code to log the stuff. I encode 998 frames (that's 1000 minus the Vdub bug :)) and get "frames: 1004"... but the cumulative framestats are perfectly the same than v1.0.1 (with 998 total frames!...). That's weird :(

I'm currently looking into it but cannot find anything wrong, the code is the same which has always perfectly worked with the 1.0.x branch. Are you guys sure nothing's wrong in the VfW frontend eh? (example: the bitrate calc makes the GUI crash, for example...)

[EDIT] never mind, fixed. I was just counting the dropped frames too. That damn VfW interface! :D (but the PSNR is still the same, slightly lower than v1.0.1)

more later ;)
virus

[SeeMoreDigital]

Originally posted by Andrey
It would be great if sysKin posts it here, when he thinks he is done with b-frames. It would be interesting to test new code... Arhhhhh.... XviD B-VOP!

[virus]

ok, here's my useless effort :)

I see a couple of interesting things that may help sysKin fix what's broken, so I'll post it anyway.

Versions:
XviD 1.0.1: Koepi's build + custom xvidvfw.dll
XviD CVS July 20, 2004: Gamr's 200407201540 build + custom xvidvfw.dll (just a quick hack of the xvid_20040720 daily snapshot :D)

Settings:
prof. unrestricted, quant MPEG, QPel, AQ, BVOPs 2/1.50/1.00, Closed GOV, no GMC, no PB
MSP 6, VHQ 4, Max I-VOP int. 250, Chroma motion, Trellis, I/P/B q. 2-31
2nd pass: I-VOP boost 10%, reduction 1/20%, overflow 10/10/8, curve compression 0/0
VHQ for B-frames enabled for XviD CVS

Clip:
1000 frames, 560x304 @ 25fps, clean DVD source ("Ghost Ship", PAL, chapter 3), high motion
1st pass size (1.0.1): 2394.4 kbps (PSNR 46.03 dB)
1st pass size (CVS): 2416.6 kbps (PSNR 45.99 dB)

Frametype distribution (both versions):
I-frames: 32 (3.2%)
P-frames: 520 (52.1%)
B-frames: 446 (44.7%)

AVS script:
Mpeg2Source("c:\dvd\gs\ghost_ship.d2v")
LanczosResize(560,304)
Trim(7950,8949)

Encoding in VDub 1.5.10 (Fast recompress)
Note: due to the well-known VDub bug, the encoded frames are just 998.

average PSNR vs. bitrate

XviD 1.0.1
nominal rate - effective rate - PSNR (dB)
2100 - 2095 - 45.98
2000 - 1995 - 45.89
1900 - 1895 - 45.81
1800 - 1795 - 45.71
1700 - 1695 - 45.60
1600 - 1595 - 45.47
1500 - 1495 - 45.33
1400 - 1380 - 45.16
1300 - 1280 - 44.92
1200 - 1179 - 44.68

XviD CVS (July 20)
nominal rate - effective rate - PSNR (dB)
2100 - 2095 - 45.98
2000 - 1995 - 45.88
1900 - 1895 - 45.78
1800 - 1795 - 45.68
1700 - 1695 - 45.55
1600 - 1594 - 45.41
1500 - 1483 - 45.31
1400 - 1383 - 45.12
1300 - 1281 - 44.87
1200 - 1182 - 44.57

As you can see, the more you compress, the more things get worse. Also the 1st pass size difference shows there's something wrong in the current code I think ;)

hope this helps :)
virus

[Sharktooth]

Syskin said something is broke in bframes code...
Its perfectly normal if you rise the bitrate the difference will be small or none coz quantizers will lower until the Q.matrix get staurated and bframes will be less important...

[chilledoutuk]

PSNR corect me if im wrong is a measurment that compares frames and measures the signal to noise ratio between it and the source.

What i have noticed is the more acuarate and fluid motion etsimation where bframes are used i dont think that psnr takes this into account.

[Sharktooth]

Can you explain how you noticed a more accurate and fluid motion EXTIMATION?
Maybe you noticed a more accurate and fluid motion (not more than the source...).
However PSNR will measure ANY difference between source frame and compressed frame.
That means if the motion extimation algorithm is broken or works better it will be reflected in a difference in the encoded frame and PSNR test will measure it.

[sysKin]

It seems that one reason for the difference has been found. It's still unclear if 1.1's bframes give better results compared to 1.0.1's bframes (well, they are much faster) but with VHQ, they are better.

Have fun testing,
Radek

[virus]

Tried the very latest Gamr's build (200407212350), carrying the new fixes, on the same clip: the 1st pass size is somewhat reduced (2413.5 kbit/s, PSNR 45.99 dB), but the average PSNR curve is still within ~0.01-2 dB from the values found yesterday (higher or lower, it depends).

cheers :)
virus

[obieobieobie]

Awesome. I like the sound of better bframes. :)

[chilledoutuk]

Sharktooth I dont like your attitude all I was saying is visuly to me the new vhq for bframes seems to improve the smothness of the motion in the video.

The ultimate test is a subjective Human visual one as thats why your encoding videos in the first place. The new vhq reduces blocking artefacts. a slightlly less detailed image to humans is normally prefrable than visable macro blocks.

By the way does PSNR measure motion compensation?

Also PSNR does not take into consideration the Persistence of the HVS which the codecs do.

[Sharktooth]

Originally posted by chilledoutuk
Sharktooth I dont like your attitude
Well, i dont care that much... however if you feel offended in some way here's my apologies: i'm sorry.
all I was saying is visuly to me the new vhq for bframes seems to improve the smothness of the motion in the video.
You said Motion ESTIMATION.Motion extimation is a phase during encoding. There is no way you can notice (how?!?) a better motion ESTIMATION during the playback. All you can notice is a better motion.

The ultimate test is a subjective Human visual one as thats why your encoding videos in the first place. The new vhq reduces blocking artefacts. a slightlly less detailed image to humans is normally prefrable than visable macro blocks.
Here we can discuss for ages... There is ppl who likes detail even if there are blocks and there is ppl who likes the smoothness in place of artifacts. Personally i like more the VP6 approach...


By the way does PSNR measure motion compensation?
Also PSNR does not take into consideration the Persistence of the HVS which the codecs do.
PSNR measures the difference between the source and the encoded frame. Starting with the fact that the BEST possible encode is to have exactly the same immage as the source frame then it is obvious the more the compressed frame looks like the original the better is the encode.
In the case of a bad motion compensated frame it will look different from the original, hence the measured PSNR will be (theoretically) lower than optimal.
However all metric tests on the single frames have been proved to be not an absolute reference for quality. But for example average PSNR, overall PSNR and SSIM tests are a much better numerical representation of quality.

[chilledoutuk]

Yes sharktooth I made a typo I’m so sorry but that’s no excuse to be obnoxiously pedantic.

As you know subjective improvement in the smoothness of motion is normally a resultant of more accurate motion estimation, I’m sorry if I lost you with this comment (perhaps my typo suddenly turned you into a retard).

PSNR is useful for showing numerically the noise generated by the compression process with reference to the source frame that’s it.

The problem with PSNR is that it does not asses the quality of video sequences (yes sequences not individual frames) the same way as the HVS (Human Visual system) does.

It’s the deficiencies of the HVS that allow us to lossly compress video whilst remaining largely ignorant to the information lost.

As you undoubtedly know it is not possible to loosely compress anything without loosing information. The trick is to loose information that is less likely to me missed, Missed by the HVS not some mathematical equation.

Idealistically compressed video frames would have no image information lost. But this is obviously impossible for a lossy compression algorithm to achieve.

ALL lossy video compression algorithms take the characteristics of the HVS into consideration and target the distribution of information to where maximum subjective quality to the HVS will be achieved, sometimes at the detriment of PSNR.

Bottom line is PSNR considers all image information to have equal importance where the HVS does not.

[Bulletproof]

Does VHQ work on I-frames as well?

[sh0dan]

No.

[OCedHrt]

I did an encode of the entire movie for Pirates of the Caribbean with the new BVOP code and there doesn't seem to be any noticeable problems at least :P I don't really know how to compare which is better so I leave that up to the regular testers :D

But I do have a quick question with the newer builds on gamrdev, although not really related to the BVOP issue but I don't think it warrants its own thread.

On the newer builds I see that there have been changes to the dshow filter, but the dshow filter xvid.ax that I get installed with the gamrdev build has been the same one forever, modified 4/30/2004. Is this normal?

[celtic_druid]

Think it includes a pre-compiled binary, although as far as I know the dshow filter is now compilable via gcc so it could be included.

All my builds include a matching xvid.ax so you could always just download the 7z and copy it over.

[OCedHrt]

Where would I be able to obtain your builds? :D Nvm, did a search and found it :) Thanks! Wow, even has to auto AR :)

[sysKin]

Originally posted by OCedHrt
Wow, even has to auto AR :) Btw, would be nice if you checked THAT. It works on some players but doesn't work on some others...

[SeeMoreDigital]

Hello everyone,

Can anybody here provide some short 1B-VOP and/or mB-VOP VHQ samples (without GMC and Qpel) so I can test them in hardware please?


Thanks

[sysKin]

There is no way VHQ will affect your hardware - it's not a profile setting, it's encoder complexity setting.

[SeeMoreDigital]

Originally posted by sysKin
There is no way VHQ will affect your hardware - it's not a profile setting, it's encoder complexity setting. All the same... it would be interesting to view them in hardware on my various collection of monitors and equipment ;)


Cheers

[chilledoutuk]

celtic_druid i have been using your builds for ages there as fast as koepis but as upto date as gamr's which means i can test the latest code without suffering a performance hit.

http://celticdruid.no-ip.com/xvid/

to save seraching

cheers

[OCedHrt]

SeeMoreDigital: I can provide one of those crazy sample clips I've been testing :P

[SeeMoreDigital]

Originally posted by OCedHrt
SeeMoreDigital: I can provide one of those crazy sample clips I've been testing :P Please do :D


Cheers

[OCedHrt]

Not really a full quality one but it should do :)

Some weird settings were used:
2-pass
HVS Best
Adapative Quant
B-VOPSs 3,1.25,1.00 Packed Bitstream, Closed GOV
Chrmoa optimzer
BVOP sensitivity 10
Motion search 6
VHQ 4
BVOP VHQ
Turbo
Trellis

Decided to try some slightly more aggressive bvop settings to see if the VHQ will make up for the degrade? in quality.

Seems to be pretty good for 5 mb :P

http://s90713212.onlinehome.us/images/pirates.mp4

If this shouldn't be here I'll take it off, but off to sleep for now :)

[SeeMoreDigital]

Thanks for the encode.

I managed to get your encode to work but only I after de-muxing the video stream to AVI and removing the packed bitstream using MPEG4 modifier.

After I did this the stream worked fine in hardware in either the AVI container and when muxed into the MP4 container using GraphEdit/3ivx.

Could somebody generate and post another encode but this time without packed bitstream and using a full DVD pixel frame size (ie:720x480 or 720x576)... And with as many B-VOP's as you dare (but again without GMC and Qpel).


Cheers

[*.mp4 guy]

thanks muso for the ftp server:D

[edit] links are confirmed not to work, dont bother with them.
thank you SeeMoreDigital for the help.

settings: hvs, 4bvop, no packed bitstream, trellis, motion 6, vhq4, bvhq, singlepass quant7 bvop quant 11, chroma optimizer

url:ftp://upload@mosu.no-ip.com/trailer2.avi

ps: Im not at my house so its the only high quality clip i could find to encode:(

[edit]: er someone tell me if the link is working...

if not go to:ftp://mosu.no-ip.com/
and use the user name upload and the password only then download trailer2.

[SeeMoreDigital]

Hi *.mp4 guy,

unfortunately I'm unable to download 'trailer2' using either of the methods provided :(


Cheers

[*.mp4 guy]

drat... well i guess this whole posting samples thing is rather dificult.:confused:

anyway the second option works for me, but not the first. o well it was worth a try.;)

[SeeMoreDigital]

Just tried the second method again...

I can see the encodes but I don't have permission to copy/save them...

http://img46.exs.cx/img46/8503/SMD_ftp_drop.gif


Cheers

[Sharktooth]

Well, i suspect that account is for UPLOAD ONLY :D

P.S.:
@chilledoutuk: i dont want to start a flamewar so i didnt even read your post. have fun in your ignorance.
thank you drive thru.

[SeeMoreDigital]

Originally posted by Sharktooth
Well, i suspect that account is for UPLOAD ONLY :D Bummer!

Could anybody else oblige by generating and posting an encode: -

"but this time without packed bitstream and using a full DVD pixel frame size (ie:720x480 or 720x576)... And with as many B-VOP's as you dare (but again without GMC and Qpel)".


Cheers

[lordadmira]

Originally posted by Sharktooth
You said Motion EXTIMATION.Motion extimation is a phase during encoding. There is no way you can notice (how?!?) a better motion EXTIMATION during the playback. I have a question, what is "extimation"? It doesn't seem to be in the dictionary (http://www.m-w.com/cgi-bin/dictionary?book=Dictionary&va=extimation)....

LA

[Selur]

he probably ment "motion estimation" ;)

[Sharktooth]

Fixed.

[chilledoutuk]

First sharktooth now you please stop being so pedantic or at least stop posting pedantic remarks.


what are you refering to sharktooth in your minimalist coment of "Fixed"

[SeeMoreDigital]

Originally posted by chilledoutuk
...what are you referring to sharktooth in your minimalist coment of "Fixed" Sharktooth was referring to his post dated 21st July 2004 20:34...

Okay.... with that out of the way, I would respectfully ask both of you to pull yourselves together and pack this nonsense in...

You're in danger of spoiling what is a very worthwhile thread...


Cheers

[Sharktooth]

There are no problems on my side.
However im just doing some encodings with 1.0.1 and CVS Head build by celticdruid.
Usual source: The Animatrix.
I'll upload them as soon as i finish.

[SeeMoreDigital]

Originally posted by Sharktooth
...Im just doing some encodings with 1.0.1 and CVS Head build by celticdruid.
Usual source: The Animatrix.
I'll upload them as soon as i finish. Many thanks ;)

[Sharktooth]

Uhm... new build up... re-encoding *grrrrr*

[Sharktooth]

Done:)

Here's the clips:

Xvid Head build 25/07/2004 (Celticdruid's Build) (http://valborg.hn.org/sharktooth/xvid_25072004head-BVHQ-2000kbps-6of9hvs.avi)

Xvid 1.0.1 release (Koepi's build) (http://valborg.hn.org/sharktooth/xvid-2000kbps.avi)

Notes:

1024x432, 2000kbps, MSP 6, VHQ 4, GMC, AQ, CM, CO, TURBO, BVOPS 2/1.50/1, I-FRAME int. 250, Trellis, MinQ/MaxQ 1/31, 6of9-HVS.

Same settings for both encoders except VHQ for B-Frames enabled for the Head Build.

EDIT: Uploads completed.