AutoQMatEnc

[Axlemar]

As normal for me, the current one I am trying is fairly high bitrate animation: Reduction Level for DVD-5: 64.8%
- Overall Bitrate : 4,147Kbs
- Space for Video : 3,899,814KB
- HIGH/LOW/TYPICAL Bitrates: 4,514/600/4,147 Kbs (Last Exile). I am using the newest rebuilder and all the stable other. I have noticed that the first couple of passes are really low (.6 ratio) but if I set the passes to like 6 it slowly gets to the .9 tange.

[SAPSTAR]

As normal for me, the current one I am trying is fairly high bitrate animation: Reduction Level for DVD-5: 64.8%
- Overall Bitrate : 4,147Kbs
- Space for Video : 3,899,814KB
- HIGH/LOW/TYPICAL Bitrates: 4,514/600/4,147 Kbs (Last Exile). I am using the newest rebuilder and all the stable other. I have noticed that the first couple of passes are really low (.6 ratio) but if I set the passes to like 6 it slowly gets to the .9 tange.
Oky doc...it means I still have to work on high bitrates animations :) ...
I'll try to work on it before my vacations...

[alfixdvd]

AutoQmatEnc 0.26b.

File AutoQmatEnc.exe has file size of 92K and Date 07/12/05

When it runs it shows

C:\DvdTools\AutoQMatEnc026b>AutoQMatEnc.exe

AutoQMatEnc V0.25, 05/07/05, SAPSTAR - www.Doom9.org
Usage: AutoQMatEnc.exe -ecl item.ecl


This is It correct ?

[SAPSTAR]

AutoQmatEnc 0.26b.

File AutoQmatEnc.exe has file size of 92K and Date 07/12/05

When it runs it shows

C:\DvdTools\AutoQMatEnc026b>AutoQMatEnc.exe

AutoQMatEnc V0.25, 05/07/05, SAPSTAR - www.Doom9.org
Usage: AutoQMatEnc.exe -ecl item.ecl


This is It correct ?
Hmmm yep...I forgot to update that part ! ;) Thx for pointing it ! To be sure, check the product version in the properties. (right-click on it)

[alfixdvd]

Hmmm yep...I forgot to update that part ! ;) Thx for pointing it ! To be sure, check the product version in the properties. (right-click on it)

Yes in properties 0.26.0.4

Thanks Sapstar

[SAPSTAR]

Hi....I think this time I have a really good track...faster and better should be the next version ;)...and depending on the speed of the beta testers, it could be released really soon.....

[lunario]

sapstar i want to use your encoder but untill now nothing.... i´ve created the ecl with eclcreate

i put autoqmatenc -ecl movies.ecl and then it says this

E:\temp2\Movie0.avs: 704x480, 15712911/524288 fps, 1392 frames
converting YUY2 -> YV12
Assigned new YUV clip
converting YUY2 -> RGB24
Assigned new RGB clip
Creating : C:\Intra0.ppm

and then nothing..

what i am doing wrong ? does automatenc needs dvd rebuilder to work ?

[SAPSTAR]

sapstar i want to use your encoder but untill now nothing.... i´ve created the ecl with eclcreate

i put autoqmatenc -ecl movies.ecl and then it says this

E:\temp2\Movie0.avs: 704x480, 15712911/524288 fps, 1392 frames
converting YUY2 -> YV12
Assigned new YUV clip
converting YUY2 -> RGB24
Assigned new RGB clip
Creating : C:\Intra0.ppm

and then nothing..

what i am doing wrong ? does automatenc needs dvd rebuilder to work ?
Not at all....but something is weird look at the fps detected by avisynth :
it's a 29.97fps ratio but...looks weird...did you check if your avs file is playing well ?! What version of Avisynth are you using ?!

[lunario]

i am using latest beta .. and yes the source avi has 29.97 ... i will try with 2.55 and i will give you my input

[lunario]

ummm now it works... ummm i guess the prblem was the source 29.97 i made the avs with diko now with another movie and it works...

[lunario]

it was the source ... because i´ve installed latest avisynth and it works

ps i am using avisynth 2.56 beta 4 again with another source and is working

i can´t wait to see the quality of your encoder :)

[SAPSTAR]

it was the source ... because i´ve installed latest avisynth and it works

ps i am using avisynth 2.56 beta 4 again with another source and is working

i can´t wait to see the quality of your encoder :)
I noticed some strange behavior with the latest version 0.26, I would recommend using the 0.24b until I release 0.27b this Friday or this weekend...I identified the pb I introduced after 0.24 n the rate control...some spikes could occur....In the current 0.27b I'm testing everything is perfect up to now !

[MaDxCrEaM]

must of ran into some problems?

[hutch1711]

must of ran into some problems?
The poor guy has 3 children under six years of age, was planning on going on vacation, and now us poor snowbirds to the north are wilting under a two week tropical heat wave, that has tempatures in the 30's celsius with humidex readings in the 40's. If he's smart he's sitting in front of an air conditioner and downing a few cold ones. Knowing Sapstar he just want's to get it right before he releases next version. :D

[Carpo]

If he's smart he's sitting in front of an air conditioner and downing a few cold ones. Knowing Sapstar he just want's to get it right before he releases next version. :D

nah hes in the pool most days :D

[MaDxCrEaM]

The poor guy has 3 children under six years of age, was planning on going on vacation, and now us poor snowbirds to the north are wilting under a two week tropical heat wave, that has tempatures in the 30's celsius with humidex readings in the 40's. If he's smart he's sitting in front of an air conditioner and downing a few cold ones. Knowing Sapstar he just want's to get it right before he releases next version. :D

LOL, in no way I was rushing him. Sapstar nor his testers haven't responded for a bit so I thought I would try to get some info :)

[Axlemar]

He is still working hard on getting a new version out, so don't worry.

[Carpo]

im testing 0.27b3 atm - thats why we not responding - busy testing like mad to get new version out (with a bit of luck) before sapstar goes on vaction :D

[SAPSTAR]

Hi there,

This is most probably my last release for the next 2 weeks....
0.27b - 2005/07/20
- Added a low motion detector to tweak the rate control
- Improved the rate control
- Updated the averaging in QMatOp
- Corrected an error on very big files for the Nth path(N>2)
- Minor tweakings and errors.

There are still issues with the high bitrates animations, outside this type of movie it should be ok....Good Encode !!!
Special thanks to Axlemar and Carpo for their tests of this last week....
Last minute info : The 1webspace.org web site died...my primary web site will be now the one provided by badservo : www.badservo.com/autoqmatenc

[hutch1711]

This is most probably my last release for the next 2 weeks....

Thank you Sapstar. Enjoy your vacation with the family. :)

[MaDxCrEaM]

Thanks Sapstar, look forward to testing this out.

[CorrsFan]

I just had some major problems with the Firefly episode "Safe." I had some macroblocking in the first segment from other encoders and was trying to find how best to encode it. I prepared the disc in DVDRB-Pro 1.00RC2 with the settings specified in the readme.txt and enabled "CCE Adaptive Quantizer Matrices."

The first time I ran into the problem was in the DVDRB Matrix Editor. I selected the first segment and "No Matrix," then prepped and encoded. These were some serious problems, ones that I've never come across before.

I've already tested AQE on a minorly troublesome segment from another disc -- seg 21 of Indiana Jones and the Last Crusade and AQE seemed fine then.

I reran everything, deleting the temp files and double-checking all settings as well, and did a straight prep and encode in DVDRB itself and it happened again. I viewed the source files themselves and they are fine and they haven't been touched since being ripped to my HD via DVDDecrypter.

I actually found a worse occurrence earlier in the segment. When there's a sequence of them, it happens every other frame -- the frames in between are fine -- and it always when there's motion and panning.

I've attached 3 images and the matrix file. I can provide more images if needed (and the log, too, but it's not very interesting).

[Axlemar]

Your attachments are not approved yet and you say that you are having major problems, but you don't say what kind. What problems are you having? What type of issue is it, what does it look like? What version are you using? Sapstar just released a new version.

[CorrsFan]

Your attachments are not approved yet and you say that you are having major problems, but you don't say what kind. What problems are you having? What type of issue is it, what does it look like? What version are you using? Sapstar just released a new version.

Huge blocks are appearing. I'm using the latest version of AQE -- it's the only one I've ever installed. Here's a pic (hopefully not too big):

http://img329.imageshack.us/img329/363/aqeoops72cc.jpg (http://www.imageshack.us)

The next frame is fine, but the one after that is like this.

Latest version of AQE -- it's the only one I've ever installed. I tried encoding with HC and it's working just fine.

Some thumbnail links to more frames:
http://img314.imageshack.us/img314/9999/aqeoops98dh.th.jpg (http://img314.imageshack.us/my.php?image=aqeoops98dh.jpg)
http://img314.imageshack.us/img314/6099/aqeoops13ut.th.jpg (http://img314.imageshack.us/my.php?image=aqeoops13ut.jpg)
http://img314.imageshack.us/img314/93/aqeoops64kb.th.jpg (http://img314.imageshack.us/my.php?image=aqeoops64kb.jpg)

[Axlemar]

In my experience this program prevents macroblocks quite well. How many passes did you do? At this point I would recommened four as the rate control still needs to be tweaked a bit. Also, I would recommended that after you run the prepare phase you go into the output folder and manually edit every .avs file and remove anything pretaining to ConvertToYUY2() or ConvertToYUY2(interlaced=true) because I personally had problems with motion with this setting and AutoQMatEnc does the conversions itself (just removing ConvertToYUY2 in rebuilder won't work because AutoQMatEnc mimics the CCE ecl and when you select CCE as the encoder ConvertToYUY2 is added automatically. I will discuss this more with Sapstar in the future). Did you add CCEAQM=1 under [options] in the rebuilder.ini to ensure that the QmatOp part of the encoder ran and does it say Adaptive Quantizer matrices enabled in the status window in rebuilder? Finally, players that hate to have each gop optimised can playback blocky. From what I have heard an XBOX is pretty loose to the mpeg-2 standard or you could just try another player if you have one. Did you try to use the encoder default matrix or are you using modified ones? And lastly you can try to burn the dvd at a slower speed or on different media. Read errors can cause display issues. If you try all these and still have problems than let me know and I think I can get my hands on firefly.

[CorrsFan]

Hmm, I was under the impression that the matrix you select doesn't matter for AQE (a previous test with Last Crusade yielded what seemed to be identical results with No Matrix -- that should mean encoder default, right? -- and SixOfNine selected) so I had "No Matrix" selected when I first encountered this problem in the Matrix Editor and "encoder default" when I did the prep & encode in DVDRB itself to verify.

But I just did an encode with the MPEG Standard matrix (and that CTY2() line removed) that FIXED the problem, but then I tried a test with the Med-Med matrix that came with the Matrix Editor that resulted in the same problem COMING BACK.

Oh, and I am doing 4 passes & have CCEAQM=1. I'm using PowerDVD, Media Player Classic, and VirtualDubMod to view the files and there hasn't been any difference between them.

BTW, thanks for the assistance, I surely do appreciate it.

[Axlemar]

What matrix you select guides some of what the encoder tries to determine. Supposedly the encoder will come up with the best matrix regardless of what you select, but it seems that maybe there are some conflicts with certain matrices and the encoder? I never use the matrix editor, I just have the default matrix settings in rebuilder and run encodes that way. I guess you should just experiment and report your findings to Sapstar in two weeks.

[Axlemar]

For people who would like to help test the encoder, I would recommended that you remove all forms of convertToYUY2 from the avs files, do four passes, and back up movies that are at least into the 5 gig range. Also, please let me know if there are any movies you back up that skip on very low or very high action scenes as I am trying to find out if there is a potential problem. Thanks.

[MaDxCrEaM]

Think I ran into a problem. I just viewed the first two segments done, the first m2v file loooks great, with the bitrate around what it should be. I viewed the 2nd m2v file, and the bitrate is averaging around 1mb looks like, when the series I'm doing is suppose to be around 2.9mb average. Here is the last part of the avs.log file for that segment:

Normalized matrix
8 14 16 19 22 22 52 58
14 14 19 20 22 24 58 60
16 19 22 22 24 54 60 62
19 19 22 22 41 58 61 64
19 22 22 29 56 59 63 73
22 22 24 53 59 62 76 93
22 22 51 58 62 74 92 107
22 46 58 62 70 88 106 117
16 17 18 18 19 20 21 22
17 18 18 19 20 21 22 24
18 18 19 20 21 22 23 25
18 19 20 21 22 23 25 26
19 20 21 22 24 25 26 27
20 21 22 23 25 26 29 34
21 22 23 25 26 28 35 40
22 23 24 26 27 34 40 45
Darkness level : 0.268002
Qmin : 1 ; Qmax : 41
Done
Pass 2/3
Bitrate : 2839000
H:\TEMPER\D2VAVS\V01000100001002.avs: 720x480, 23976/1000 fps, 41537 frames
converting ? -> RGB24
Assigned new RGB clip
AVS File opened successfully
8 14 16 19 22 22 52 58
14 14 19 20 22 24 58 60
16 19 22 22 24 54 60 62
19 19 22 22 41 58 61 64
19 22 22 29 56 59 63 73
22 22 24 53 59 62 76 93
22 22 51 58 62 74 92 107
22 46 58 62 70 88 106 117
16 17 18 18 19 20 21 22
17 18 18 19 20 21 22 24
18 18 19 20 21 22 23 25
18 19 20 21 22 23 25 26
19 20 21 22 24 25 26 27
20 21 22 23 25 26 29 34
21 22 23 25 26 28 35 40
22 23 24 26 27 34 40 45
Darkness level : 0.268002
Qmin : 1 ; Qmax : 41
Done
Pass 3/3
Bitrate : 2839000
H:\TEMPER\D2VAVS\V01000100001002.avs: 720x480, 23976/1000 fps, 41537 frames
converting ? -> RGB24
Assigned new RGB clip
AVS File opened successfully
8 14 16 19 22 22 52 58
14 14 19 20 22 24 58 60
16 19 22 22 24 54 60 62
19 19 22 22 41 58 61 64
19 22 22 29 56 59 63 73
22 22 24 53 59 62 76 93
22 22 51 58 62 74 92 107
22 46 58 62 70 88 106 117
16 17 18 18 19 20 21 22
17 18 18 19 20 21 22 24
18 18 19 20 21 22 23 25
18 19 20 21 22 23 25 26
19 20 21 22 24 25 26 27
20 21 22 23 25 26 29 34
21 22 23 25 26 28 35 40
22 23 24 26 27 34 40 45
Darkness level : 0.268002
Expected size : 77929007 ; Real Size : 614835118 ; Error : -6.9e+002 %
Qmin : 7 ; Qmax : 323
Done
Final Result :
Expected size : 77929007 ; Real Size : 207714952 ; Error : -1.7e+002 %

[Carpo]

converting ? -> RGB24 - that bit dont look right

under options->avs options - make sure both options are unticked especially ConvertToYUY2() - that is only needed (as is the other option there for cce 2.50) for cce - autoqmatop may mimic cce but it doesnt need either of those options ticked

About matraices - if you grab version 2.0.05 of rockas matrix editor it has 8 new matrices there for verious bitrates - you might wanna see if they help

also as i have been discussing with sapstar - if you use 4 passes on your encodes the sizing is bang near spot on - this is not the best work around till he can have a look at the code but it should be ok for the next few weeks while hes away on vacation

[MaDxCrEaM]

I'm testing Matrix Editor as I type. As for those options under avs, convertToYUY and the AudioDub(BlankClip) they were ticked. I never touched those before ever, so they must be ticked by default. So I unticked them. So you are saying I never need to tick those anymore with CCE 2.70.2.0? How about HC encoder? Thanks for the help.

[Axlemar]

YOU CAN'T JUST UNTICK ConvertToYUY2 BECAUSE SELECTING CCE AS THE ENCODER FORCES IT TO BE ADDED. As I have said, you have to edit the .avs files after prepare and remove it until Sapstar changes it. You most likely will get weird results with it on.

Personally I run a lot of passes because if the bitrate is really close it will skip the rest of the passes from the point on which the rate was good. I personally don't mess with the matrices because, in theory, it shouldn't matter too much. I probably should though because it seems to have more influence on the results than I previously thought.

Also, expect that when you jump to a chapter the chapter will play a couple seconds off. This is because Sapstar hasn't incoporated a closed GOP system for the encoder yet. Other than that, and some playback issues on some of the animation I have done, the encoder is designed to be very high quality and has a really intelligent approach to determining optimal values. I believe that once it is perfected it will be a very nice encoder, so please continue to try it out and let us know.

[Carpo]

I'm testing Matrix Editor as I type. As for those options under avs, convertToYUY and the AudioDub(BlankClip) they were ticked. I never touched those before ever, so they must be ticked by default. So I unticked them. So you are saying I never need to tick those anymore with CCE 2.70.2.0? How about HC encoder? Thanks for the help.

no if using cce 2.6+ (on intel) u only need to keep convertToYUY2 as cce does need that if you use cce 2.50 (on amd) u'll need both ticked

if you use HC yes both can be unticked as dvd rebuilder will put convertToYUY12 in its place which works fine with HC (well has the 10+ times i have done encodes with it :))

[Carpo]

YOU CAN'T JUST UNTICK ConvertToYUY2 BECAUSE SELECTING CCE AS THE ENCODER FORCES IT TO BE ADDED. As I have said, you have to edit the .avs files after prepare and remove it until Sapstar changes it. You most likely will get weird results with it on.

well thats strange as i do have it unticked and dvd-rb sticks in convertToYUY12 - think that was the line - and i have checked with sapstar about it and he says it should work just fine and i have done a few encodes on it

[MaDxCrEaM]

I would love to to help test AutoQMatEnc, so I tried again on the series I did earlier with undersizing of the 2nd segment using 3 pass. I used 6 pass this time, and the 1st segment was still OK as before, but the 2nd segment came out way Oversized this time using more passes. The average bitrate was over 4mb, which it should be around 2.9 mb. The 3 pass gave me around 1mb average only on the 2nd segment, so I am using CCE with using one of the new 8 matices included with Matrix Editor.

I only have one machine to do all my stuff, encoding, surfing with, and I need to get some of these encodes done. I would love to keep on doing test encodes if I had another machine (hopefully soon). I will have to wait till Sapstar is back from vacation and gets this great codec a little more fine tuned.

[Axlemar]

well thats strange as i do have it unticked and dvd-rb sticks in convertToYUY12 - think that was the line - and i have checked with sapstar about it and he says it should work just fine and i have done a few encodes on it

From the animation encodes I have done, not editing out ConvertToYUY2 has caused color issues on heavy motion scenes.

About sizing issues, the only two things I can recommend is to choose sources that are not really high bitrate and try using a iDCT thats more accurate, like the 64-bit ones, because I seem to get more accurate sizing with them. Also, if you have the time, use a huge number of passes because if the pass is ever right on then it will skip the rest of them.

[Carpo]

From the animation encodes I have done, not editing out ConvertToYUY2 has caused color issues on heavy motion scenes.

About sizing issues, the only two things I can recommend is to choose sources that are not really high bitrate and try using a iDCT thats more accurate, like the 64-bit ones, because I seem to get more accurate sizing with them. Also, if you have the time, use a huge number of passes because if the pass is ever right on then it will skip the rest of them.

ye animation is a bit diff - 4 passes is enough - as you say if the sizing is right it will skip the other passes i find that adding ,idct=7,cpu=6 to the avs files seem to be good enough as does the new matrices in matrix edito from roccas

[Axlemar]

I tried to use a different matrix other than encoder default, med-low from the matrix editor, and both the matrix default and med-low come out with the same matrix and the same filesize. Are people getting different results from the same segment using AutoQMatEnc?

[CorrsFan]

I tried to use a different matrix other than encoder default, med-low from the matrix editor, and both the matrix default and med-low come out with the same matrix and the same filesize. Are people getting different results from the same segment using AutoQMatEnc?

Other than my problems listed above, no. I've always had the same filesize and matrix, and the video always looks the same (at least when AQE is working correctly ;) ). That's what makes my problem so odd.

[SAPSTAR]

0.28b - 2005/08/05
- Corrected a major bug in the credits "type" scenes
- Corrected the rate control, 2 passes are enough for 90% of the scenes,
3/4 passes for the rest.
- Improved the sizing - Should be ok now.
For DVDRB Users, the default target size for DVDRB is 4.31Gb,
I suggest CCETargetSectors=2260000 or 2280000 when using AQE.
- Recompiled avcodec, I'm now using the very last version of it...
(some overflows corrections were brought with it)

Good encodes ....and let me know your results....

Special thanks to Badservo for the free hosting :thanks: and Axlemar for the many detailed tests he ran :thanks: ..

FYI : I lost my HDD during my vacations due to a storm, so I had to restart from the 0.24b version sources....I hope everything is back to normal now.

[Krizzz989]

I'm getting alot more "Good rate control no need to have one more pass...." messages, awesome.

[Carpo]

just to let you all know we are still working on it - by that sapstar is still tweaking and im still giving him errors to hunt down and find :devil:

[hutch1711]

Carpo, thanks for the update. Glad to hear Sapstar hasn't abandoned this project. :)

[SAPSTAR]

Carpo, thanks for the update. Glad to hear Sapstar hasn't abandoned this project. :)
No no don't worry not abandoned at all...just working on some fine tuning...the 0.29b should be released soon....

[SAPSTAR]

Here are the new release features/changes :

0.29b - 2005/08/18
- Added fix_gop_length support
(in order to keep the length of GOP steadier if your DVD player doesn't support it)
- Improved the rate control for very dark scenes
- Added packet_size parameter :
Used for the scene change threshold, bigger it is, less detection we have
- Changed the threshold for the Qmin increase
- Corrected a gap in the rate control correction linearity
- Corrected QMatOp, an error could be propagated on dark scenes
- Variable Dark limit for triggering several tweakings
- Corrected a bug, AQE starts next passes even if the rate control was perfect.
Thx Carpo for that one.
- Adjusted some thresholds

[Carpo]

wonder if this one is still going

[danpos]

@Carpo

Yep, take a look at here (http://www.vmesquita.com/forum/index.php?topic=4060.0). :)

Cya!

[SAPSTAR]

wonder if this one is still going
Yes it is....I'm to release a new version with corrections...I found some stuttering on some specific scenes (like a moving pine tree forest).I also improved the OPV mode.

I should release this new version today or tomorrow....BTW I need testers for the OPV mode.... :thanks:

[Carpo]

always works :devil:

[danpos]

Yes it is....I'm to release a new version with corrections...I found some stuttering on some specific scenes (like a moving pine tree forest).I also improved the OPV mode.

I should release this new version today or tomorrow....BTW I need testers for the OPV mode.... :thanks:

Hi Jérôme!! I already did several tests with new beta using a short piece of VOB and today I did a FULL encoding in OPV mode (where I did a prediction calculation using a EXCEL spreadsheet) and the results were highly satisfactory, wow ! I need to do the authoring, burn the compiled disc and so watch it in my SAP.

I'll report all results at the other board, but I already can to say that the improvements at new beta were great. ;)

Keep up the good work, mate ! ;)

Cya!