AutoQMatEnc

[Fishman0919]

Ohhhhhhhhhhhhh! I for some reason was thinking along the lines of 2 to 2 1/2 hour process. I asume QmatOp is on. Although thinking back on my couple uses of TmpgEnc, it took atleast 4 to 7 hours encoding vid clips to mpeg2 for dvd authoring. I'm running P4-M, 2.2Ghz, 500mb ram.

AQE is a great encoder and amazing on Full Disc backups and low bitrates but it's an over nite process for now..... but things do change ;)

[Carpo]

AQE is a great encoder and amazing on Full Disc backups and low bitrates but it's an over nite process for now..... but things do change ;)


no s**t - one dvd i did took 19 hours but it was 2h 30mins and i did have 4 passes with UnDot().Deen() so thats prob why i use BlindPP(cpu2="xxxxxx") now and seems to give same results and a little bit less time :)

[Fishman0919]

no s**t - one dvd i did took 19 hours but it was 2h 30mins and i did have 4 passes with UnDot().Deen() so thats prob why i use BlindPP(cpu2="xxxxxx") now and seems to give same results and a little bit less time :)

Maybe you should try something like "mpeg2source("C:\Test\test.D2V",idct=7,cpu=6)" instead but read this from the DGDecode guide

BlindPP() should not be used when the video is opened using the
MPEG2Source() function, because its postprocessing options will work better. Typically BlindPP() is used when opening the video with AviSynth's AviSource() or DirectShowSource().

cpu=6 and cpu2="xxxxxx" are really the samething

[Wheelie4]

While using AQE, during Phase II (encoding) is it possible to stop/pause halfway through and save your process, shutdown and continue the next day?

[Fishman0919]

While using AQE, during Phase II (encoding) is it possible to stop/pause halfway through and save your process, shutdown and continue the next day?

Yes, just like with any other encoder that works with DVD-RB... DVD-RB when restarted will ask you if you would like to start form the last completed segment and will continue from there..... AQE will also look to see if the QMatOp part was completed and the 1st pass and if so use them to save time.

[Wheelie4]

Great thx. :)

[Carpo]

norm use idct=3 that which sapstar said to use - used to use deen and undot, and ye i know idct 3 makes it longer but the results are soooo good :P

thought idct=7 was 3rd on the list 3 being the best - ahwell never mind i'll try it your way - cant hurt :devil:

cpu=6 - that a standard setting or is there diff ones for diff processors ?

[Fishman0919]

norm use idct=3 that which sapstar said to use - used to use deen and undot, and ye i know idct 3 makes it longer but the results are soooo good :P

thought idct=7 was 3rd on the list 3 being the best - ahwell never mind i'll try it your way - cant hurt :devil:

cpu=6 - that a standard setting or is there diff ones for diff processors ?

You can uses whatever idct... I like idct=7 because it 3rd for quality and twice as fast for me as idct=3... but instead of something like....

#------------------
# AVS File Created by DVD Rebuilder
# VOBID:03, CELLID:09
#------------------
LoadPlugin("C:\Program Files\DVD-RB PRO\DGDecode.dll")
mpeg2source("C:\DVD-RB\D2VAVS\V10.D2V",idct=7)
BlindPP(cpu2="xxxxxx")
trim(115015,123103)
ConvertToYUY2()
AudioDub(BlankClip())

use...

#------------------
# AVS File Created by DVD Rebuilder
# VOBID:03, CELLID:09
#------------------
LoadPlugin("C:\Program Files\DVD-RB PRO\DGDecode.dll")
mpeg2source("C:\DVD-RB\D2VAVS\V10.D2V",idct=7,cpu=6)
trim(115015,123103)
ConvertToYUY2()
AudioDub(BlankClip())

cpu=6 in the mpeg2source line is better then BlindPP(cpu2="xxxxxx")

[Carpo]

Y
#------------------
# AVS File Created by DVD Rebuilder
# VOBID:03, CELLID:09
#------------------
LoadPlugin("C:\Program Files\DVD-RB PRO\DGDecode.dll")
mpeg2source("C:\DVD-RB\D2VAVS\V10.D2V",idct=7,cpu=6)
trim(115015,123103)
ConvertToYUY2()
AudioDub(BlankClip())


how you get that in the mpeg2 line, its showing up about 4 lines down in mine - and i dont fancy having to mass edit the avs files :(

tried that mass file editor and its a load of s***e :angry:

[Fishman0919]

how you get that in the mpeg2 line, its showing up about 4 lines down in mine - and i dont fancy having to mass edit the avs files :(

tried that mass file editor and its a load of s***e :angry:

MassFile Editor is the only way I can think of that is easy to edit all the files :(

[Mug Funky]

textpad is great for search-and-replace'ing a big list of text files.

i just use idct=6 because it's xvid's idct which is fast and good. if you're worried about decoding differences, try this:

clip1=mpeg2source("blah.d2v",idct=3)
clip2=mpeg2source("blah.d2v",idct=6)

subtract(clip1,clip2)

if you see any glaring errors, i'll be quite surprised. there might be an occasional difference (+- 1 value on random chroma pixels), but certainly not something that's going to screw up an encode.

but i suppose if you're doing 4 passes, speed isn't a big deal :) (i like an encoder that hits the target rate exactly in 2 passes)

[Carpo]

MassFile Editor is the only way I can think of that is easy to edit all the files :(

i must be using it wrong then cus it dont edit the files right

:stupid:

[Fishman0919]

textpad is great for search-and-replace'ing a big list of text files.

i just use idct=6 because it's xvid's idct which is fast and good. if you're worried about decoding differences, try this:

clip1=mpeg2source("blah.d2v",idct=3)
clip2=mpeg2source("blah.d2v",idct=6)

subtract(clip1,clip2)


idct=7 is xvid's...idct=6 is Skal's and is the only one not IEEE-1180 compliant

Which iDCT you should use depends primarily on what CPU you have
and to a lesser degree, on how accurate an iDCT you desire. Most people will not be able to tell
the difference in quality between these algorithms but
they can be easily observed by combining the AviSynth filters Subtract()
and Levels(). All of the available options are IEEE-1180 compliant, except for
SSE/MMX (Skal).

Qualitywise: IEEE-1180 Reference > 64-bit Floating Point > Simple MMX (XviD) > Remaining iDCTs.

Speedwise: SSE2/MMX and SSE/MMX (Skal) are usually the fastest.
The IEEE-1180 Reference is easily the slowest.

From Dgindex 1.40 rc5
idct: 0 to 7 (default: 0)

iDCT Algorithm.
For more infomation on iDCTs please see Appendix B.
Please see Appendix C for supported CPUs.
- 0: Use value specified by DGIndex (iDCTs 6 and 7 unavailable in DGIndex)
- 1: 32-bit MMX
- 2: 32-bit SSEMMX
- 3: 64-bit SSE2MMX
- 4: 32-bit Floating Point
- 5: 64-bit IEEE-1180 Reference
- 6: 32-bit SSEMMX (Skal)
- 7: 32-bit Simple MMX (XviD)

From DgIndex 1.10 Final

idct : 1 to 5.
iDCT : iDCT algo used.
0 : Default value (in .d2v file)
1 : 32 bit MMX
2 : 32 bit SSEMMX
3 : 64 bit FPU
4 : 64 bit IEEE-1180 Reference
5 : 32 bit SSE2 (for P4)
6 : Skal's SSEMMX iDCT (fastest)
7 : SimpleiDCT (Very accurate iDCT from XviD project)
default = 0

[Fishman0919]

This is a Script that will make any diff's between 2 idct's very clear

LoadPlugin("C:\Test\DGDecode\DGDecode.dll")
mpeg2source("C:\Test\test.D2V")
clip1=mpeg2source("test.d2v",idct=5)
clip2=mpeg2source("test.d2v",idct=6)
Subtract(clip1, clip2).Levels(127, 1, 129, 0, 255)
ConvertToYUY2()

You can change the script to get it to work for you and change the idct's... I used idct=5 (IEEE Ref) and idct=6 (Faster but less quality) to really show the errors

[Axlemar]

I've noticed two problems that occur on all animation backups I have done and can be seen on both a dvd player and a computer. During high action scenes sometimes the colors will change for split second and then change back to their original, when in the source it stays one color. By itself it is hard to notice, but often several frames back-to-back have it and then it is easily noticable. Also, when scenes change rapidly sometimes they will flicker. The same things occurs in several different series so I don't think it is the source.

I've tried the majority of idcts' and I get the same result from all of them. The flicker doesn't annoy me too much, but the color distoration is. Any ideas?

[gilligan2]

I get an error every once in a while saying something about the bitrate being too low,and to press any key to continue.I press the key and it continues,but was wondering what would be causing it to stop.It happened last night overnight while i was doing an encode so it sat idle for 6 hours until i woke up and that was 6 hours wasted.

Anyone have any ideas ?

Next time it happens i'll give the exact error message since it got removed from the dos box after i hit the key to continue.

g:)

[SAPSTAR]

I get an error every once in a while saying something about the bitrate being too low,and to press any key to continue.I press the key and it continues,but was wondering what would be causing it to stop.It happened last night overnight while i was doing an encode so it sat idle for 6 hours until i woke up and that was 6 hours wasted.

Anyone have any ideas ?

Next time it happens i'll give the exact error message since it got removed from the dos box after i hit the key to continue.

g:)
Hmmmm without the message I can't be sure....but if you could give me the min bitrate, avg and max...it may give me an hint !

[gilligan2]

Ok,if it does it again i'll be sure to get all the info to you.

g:)

[gilligan2]

OK,here's a screen shot of what it is doing.Once i hit any key it seems to continue with no troubles on that segment.It doesn't do it all the time. :thanks:

g:)

[Axlemar]

I just got oversized results. The original is 4.96 gigs. HIGH/LOW/TYPICAL Bitrates: 8,168/943/2,952 Kbs. Reduction Level for DVD-5: 82.7%
- Overall Bitrate : 2,952Kbs
- Space for Video : 3,499,778KB.
The video came out 4.55

Also, I am going to guess why I am having the color issue. I always left the option ConvertToYUY2 on in rebuilder even though it seems that only CCE needs it. Leaving it on would probably cause color conversion to occur with two different programs (avisynth and avcodec) and that could cause the color mis-match I have been having.

[SAPSTAR]

I just got oversized results. The original is 4.96 gigs. HIGH/LOW/TYPICAL Bitrates: 8,168/943/2,952 Kbs. Reduction Level for DVD-5: 82.7%
- Overall Bitrate : 2,952Kbs
- Space for Video : 3,499,778KB.
The video came out 4.55

Also, I am going to guess why I am having the color issue. I always left the option ConvertToYUY2 on in rebuilder even though it seems that only CCE needs it. Leaving it on would probably cause color conversion to occur with two different programs (avisynth and avcodec) and that could cause the color mis-match I have been having.
FYI : I'm converting to YUV internally if not YUV.....+RGB24 for QMatOp, so yes you can remove the ConvertToYuY2, good point :goodpost: !
For the oversizing...I hope the next version will solve the sizing issues...I may release it soon....depends on the time I can spend on it !

[Axlemar]

Cool, I hope it comes out soon because I have a new batch of dvds I need to back up and I think I solved the color issue, so sizing is the only problem I have been having.

Thanks for all the hard work :D

BTW: I have noticed on my animation backups (which is basically all I backup) that the calculated matrix always starts off with really low inter and intra numbers, but that they seem to gradually go over 100. Is the animation optimization supposed to really limit the matrix values or is it normal for it to get that high?

[SAPSTAR]

0.25b - 2005/07/07
- Improved the rate control.
- Replaced the turning cross by a percentage indicator
- Log file creation : <AVSFILE>.log
- The rate control automatically detects, if it has any chance to improve the movie in the next pass if not it skips it.

The rate control is better , it may correct many undersizing issues..some may still be there. I implemented a logfile generation in order to help me in the debugging. Thanks to Carpo for this idea....

[hutch1711]

Damn! half way through an encode. Thanks as always. :D

[SAPSTAR]

Sorry I just discovered that the setup for 0.25b still contains the 0.24b version...I'll correct that as soons as possible....really sorry...
[EDIT] The package should be OK now.....I added a last minute correction I hope it will be good....

[Sedoy]

SAPSTAR your Installer change the PATH Variable from "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;" to "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;c:\Programme\AutoQmatEnc"

This is not good, because i can not start any Sysprog (reg.exe, ipconfig.exe etc) from CMD.

This is only from Installer changes, manually Change the Variable working perfekt :confused:

[SAPSTAR]

SAPSTAR your Installer change the PATH Variable from "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;" to "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;c:\Programme\AutoQmatEnc"

This is not good, because i can not start any Sysprog (reg.exe, ipconfig.exe etc) from CMD.
With ";"
Hmm strange...nobody reported that to me before....I'll check
[EDIT] in the next release I'll add this ; at the end of the path...What os are u using ?

[Carpo]

i think im losing the plot :eek:

[Sedoy]

Hmm strange...nobody reported that to me before....I'll check
[EDIT] in the next release I'll add this ; at the end of the path...What os are u using ?

I'm edit my Post.

This is only from Installer changes, manually Change the Variable working perfekt, with and wihout ";" :confused:

[Sedoy]

This Trouble its only from your Installer.

I make an Test Installer with NSIS only for Change this two RegStrings:
WriteRegStr HKLM "SYSTEM\ControlSet001\Control\Session Manager\Environment" "Path" "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;$PROGRAMFILES\AutoQMatEnc"
WriteRegStr HKLM "SYSTEM\CurrentControlSet\Control\Session Manager\Environment\Path" "Path" "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;$PROGRAMFILES\AutoQMatEnc"

And i have no Problems.

[Carpo]

i have testd all the installers all work fine for me

[SAPSTAR]

Hi all,

Just to inform you I discovered a major flaw in the rate control, for bitrates higher than 2500-3000 kps, there is a good chance for your vid to be undersized, especially for animations or dark movies.
Expect a new release during the weekend.....

[Axlemar]

Ok, thanks for letting us know.

[spyhawk]

SAPSTAR your Installer change the PATH Variable from "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;" to "%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;c:\Programme\AutoQmatEnc"

This is not good, because i can not start any Sysprog (reg.exe, ipconfig.exe etc) from CMD.

This is only from Installer changes, manually Change the Variable working perfekt :confused:Same here. Can't use system command. Have to go to system32 to use it. To eliminate this problem, just add ";" at the end of the PATH.

[alfixdvd]

Hello Sapstar the same problem with path.

I can not start any Sysprog ( c:\windows\system32 ) from CMD.

If I remove AutoMatQuenc from path, all's rigth.

If I add manually AutoMatQuenc to path, all's rigth.

My OS is Windows Xp SP2

[Carpo]

wait for new version that should be out over the weekend - its fixed

[gigah72]

would it help speedup process to add a path variable for storing dctune temp files on a ramdisk?

speed is just plain horror, quality is just incredible.
what about a qmat genereration only mode, coz the resulting qmat compared to qmatop is just to big difference to ignore in result?

[SAPSTAR]

would it help speedup process to add a path variable for storing dctune temp files on a ramdisk?

speed is just plain horror, quality is just incredible.
what about a qmat genereration only mode, coz the resulting qmat compared to qmatop is just to big difference to ignore in result?
The next version should be faster....the rate control error I found slows significantly the process too.
About the QMat generation mode only, it may come...in fact I may rewrite QMatop entirely to use the same algo than AQE, but it will be a big job !

[MaDxCrEaM]

I just got an "Could not open Vid file, press any key to continue". Not sure why, anyone else get this problem?

[Fishman0919]

I just got an "Could not open Vid file, press any key to continue". Not sure why, anyone else get this problem?

Are you using DVD-RB or are you manually encoding??

[Axlemar]

I am getting the "Could not open vid file" message also, but I am quite sure it is related to the rate control because it only happens if the expected and actual sizes are way off.

[MaDxCrEaM]

I am using DVDRB. I just press enter, and it continues on to next segment. But I stopped it since a new version will be out this weekend since an encode last time took me 29 hours ;)

[Carpo]

bet the film looked s**t hot tho :D

[Krizzz989]

I'm getting "could not open video" messages. Thanks in advance for any help.

I used eclcreate.

avs file:
#------------------
# AVS File Created by DVD Rebuilder
# VOBID:01, CELLID:22
#------------------
LoadPlugin("C:\Program Files\DVD-RB PRO\DGDecode.dll")
mpeg2source("C:\DVD-RB\D2VAVS\V01.D2V")
trim(135183,142960)
ConvertToYUY2()
AudioDub(BlankClip())

ecl file
; CCE SP Trial Version -- Encoder Control List

[item]
title=V01002100001022.AVS
vid_out=1
vaf_out=1
aud_out=0
aud_mode=0
video_type=4
vid_file0=C:\DVD-RB\V01002100001022_AVS.mpv
vid_file1=C:\DVD-RB\V01002100001022_AVS.mpv
vaf_file=C:\DVD-RB\V01002100001022_AVS.vaf
aud_file=C:\DVD-RB\V01002100001022_AVS.mpa
file_focused=0
width=720
height=480
timecode=0x1000000
tc_ref_frm=0
drop_frame=0
cbr_brate=6000
opv_q_factor=35
opv_brate_min=300
opv_brate_max=1500
vbr_brate_avg=700
vbr_brate_min=300
vbr_brate_max=1500
vbr_bias=30
vbr_pass=2
create_new_vaf=1
pulldown_detect=0
letter_box=0
panscan=0
aspect_ratio=3
credits_tweak=0
credits_start=0x00000
credits_brate=1000
h_filter=0
h_filter_idx=8
use_filter=0
filter_val=4
dither=0
dither_max=8
quality_prec=25
intra_dc_prec=0
alternate_scan=0
non_linear=1
progressive=1
gop_m=3
gop_nm=5
gop_hdr=15
seq_hdr=1
seq_endcode=0
dvd=0
all_closed_gop=0
fix_gop_length=0
fix_vbv_delay=0
offset_line=1
half_width=0
half_height=0
uscc=0
lum_level=0
packet_size=2048
frame_rate_idx=4
qmat_idx=-1
qmat_name=bvcd
qmat=
16 18 20 22 26 28 32 39
18 20 22 24 28 32 39 44
20 22 26 28 32 39 44 48
22 22 26 32 39 44 48 54
22 26 32 39 44 48 54 64
26 32 39 44 48 54 64 74
32 39 44 48 54 64 74 84
39 44 48 54 64 74 84 84
20 24 26 28 38 42 46 53
24 26 28 38 42 46 53 58
26 28 38 42 46 53 58 62
28 38 42 46 53 58 62 68
38 42 46 53 58 62 68 78
42 46 53 58 62 68 78 88
46 53 58 62 68 78 88 99
53 58 62 68 78 88 99 99
mpeg1=0
mpeg1_cps=1
brate_idx=7
stereo=2
samples_per_sec=0
crc=1
adjust_q_matrix=1

[file]
name=C:\DVD-RB\V01002100001022.AVS
type=0
frame_first=0
frame_last=7778
encode_first=0
encode_last=7778

log file (I deleted the log file, then ran autoqmatenc again, to create this log file. If you need the original log file, I can recreate it)
QMatOp feature enabled.
Pass 1/3
C:\DVD-RB\V01002100001022.AVS: 720x480, 23976/1000 fps, 7778 frames
converting YUY2 -> YV12
Assigned new YUV clip
converting YUY2 -> RGB24
Assigned new RGB clip
AVS File opened successfully
Valid Matrix Detected
Matrix Read :
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
Darkness level : 0.011676
A VAF file is there...Checking it!!!
Picture Number : 7777
VAF File checked successfully...Pass 1 skipped
Pass 2/3
C:\DVD-RB\V01002100001022.AVS: 720x480, 23976/1000 fps, 7778 frames
converting YUY2 -> YV12
Assigned new YUV clip
converting YUY2 -> RGB24
Assigned new RGB clip
AVS File opened successfully
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
Darkness level : 0.011676
Could not open Video...
Pass 3/3
C:\DVD-RB\V01002100001022.AVS: 720x480, 23976/1000 fps, 7778 frames
converting YUY2 -> YV12
Assigned new YUV clip
converting YUY2 -> RGB24
Assigned new RGB clip
AVS File opened successfully
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
24 24 24 24 24 24 24 24
Darkness level : 0.011676

[multicone]

What about matroska support for this MPEG2 encoder ? Seems like matroska team are looking for an official MPEG2 encoder partner : http://article.gmane.org/gmane.comp.multimedia.matroska.general/1282

[Beek]

SAPStar,

Wow, can't believe the quality that can be achieved using AutoqMatenc and DVD Rebuilder Pro. I am absolutely speachless. I had long since given up doing full DVD backups 7+ gb disks, as to my eye they were sub standard (especially medium dark backgrounds) . I have used many encoders, codecs, applications throughout the years, and I must say this has really got me excited.

Please keep up the develpment, and get a Donation Page up too, love to show my appreciation beyond a few kind words.

Good to hear, the next version will be a little faster, and hopefully get rid of the "Could not open vid file. Press Return To Continue" error which I get on certain segments, although in my book, I would much rather have an nice quality backup as opposed to super fast encoding time

Cheers, Steve (Ottawa, Canada).

[SAPSTAR]

Finally I release 0.26b...the rate control is not yet as perfect as I would like, but is definitely better. I release it because I'm not sure if I will be able to work on it the 2 or 3 following weeks because I will be in vacation....

0.26b - 2005/07/13
- Corrected the PATH variable, added a ; at the end.
- Corrected the rate control once again...a new approach was implemented.
- Corrected a bug when the rate control was totally wrong,
the Nth pass was ending asking for pressing a key.
[EDIT] May be faster too...not 100% sure yet !

[Krizzz989]

Awesome, now I can start backing up my DVDs again.

[Axlemar]

Sizing is still off, but it seems to get better if you do high passes (4+).

Maybe later you could have it print the bitrates for the segment in the dos window to see what specific things can make it occur?

[SAPSTAR]

Sizing is still off, but it seems to get better if you do high passes (4+).

Maybe later you could have it print the bitrates for the segment in the dos window to see what specific things can make it occur?
Could you be more precise ? Give more details...what movie, ....
[EDIT]The bitrate is written in the log itself....be sure to use 0.26b, depending on your IE settings, you could download an old version.