AutoMKV 0.93a - (DVD/TS/AVI/AVS to X264/XviD/WVC1 into MKV/MP4/WMV) update 30/12/2007

[Die's Man]

A strange behaviour to report:
Ever since I switched from "No resize" to "Bicubic" for my "whole season in a row" batch jobs, I've been having autoMKV freezing everytime at the "Checking script...movie.avs" phase.

That is, 1st job runs just fine, file generated is functional, but the 2nd one just crashes here and doesn't even start encoding.


Oh, and by the way, I just attached an archive containing the various stuff to generate queue files I talked about previously.

In a few words, the way to go is:
1) Have your episodes ripped in different folders, grouped by "DVDs" they're in.
2)Have the titles (file names) written in the "Titles" file
3)Have your "job configuration" pasted in the "job template" file
4)Have your encoding profile pasted the same way in the "profile template"
5)Run the Jobs_gen python script to have your jobs generated, after modifying the paths at the beginning to fit yours.

Of course you need the python interpreter in place if you want anything of this to run ;-)

[delacroixp]

A strange behaviour to report:
Ever since I switched from "No resize" to "Bicubic" for my "whole season in a row" batch jobs, I've been having autoMKV freezing everytime at the "Checking script...movie.avs" phase.

I had a similar experience awhile back where I ticked "Checking Script" just for fun and AutoMKV would crash, burn and die.
I've never written a script but it certainly seams incredibly powerfull... especially this Python generator thingy...
Anyway, unticking the "Checking Script" should work fine untill the 'real' bug/problem can be seen to.

You've got a nice system going.


:):devil::D
Pascal

[spotter]

your audio files have some problems

for forcing directshowsource use in advanced settings "DSS for Audio"

you can also try besweet to encode audio

BHH

I don't think its my audio files, I've just md5summed them on good and bad runs and they were exactly the same.

Is there any way for automkv to be more verbose about why a point failed ("virtualdub returned error XYZ", "could not stat file"....), part of me thinks it might be a race condition of some sort as it happens to fail more often when disk io is under load.

[delacroixp]

1) The quantitizer box/button only shows up when I'm using the pure Constant_Quality profile, but I'm not allowed to use it when
I'm encoding with any of the other constant quality profiles, like AQ_Constant_Quality or Constant_Quality_QP.
I was just wondering if that was normal... Are the other ones not real constant quality profiles?

Quite normal though the other CQ profiles are a little "too real". CQ_CRF refers to perceived CQ with a better understanding of human psychology/biology limitations. The other CQ profiles are more strict in their approach and maintain absolute CQ without deviation (it's like encoding 64-bit colour for a Zebra which only sees in black and white).
Of course, you may have exceptional human beings with outstanding powers of sensory perception... CQ_CRF merely maximises the average to achieve evermore "lean and mean" filesizes.

To a certain extent I've given up working with the Constant_Quality.xml file since it's easier to enter your quantizer values into a Settings.ini profile which overides any quantizer value in the *.xml encoder profile. You can edit them with WordPad as need be.
If you 'Save settings' on the front page... a similar profile is saved under 'AutoMKV.ini' and will be used by default when you restart AutoMKV.
(usefull in remembering paths to input/output folders, virtual drives and stuff)

I gather that buzzqw is finetuning the system to more easily enter quantizer values for 'renamed' constant-quality profiles.


:):devil::D
Pascal

[delacroixp]

part of me thinks it might be a race condition of some sort as it happens to fail more often when disk io is under load.
I've had similar problems with VirtualDub reporting memory violations (http://forum.doom9.org/showthread.php?p=1071060#post1071060) .
Over-reaching the capacity of your HDD may cause spurious errors but over-extending the CPU with excess multi-tasking while the encode is running may also cause issues... some less obvious than others.


:):devil::D
Pascal

[spotter]

I'm thinking it can also be related to the anti-virus (in this case, symantec endpoing protection, their business class anti-virus), perhaps the anti virus having some sort of lock on the file that prevents automkv from verifying that its ok, and due to load it being held longer than it should be.

[spotter]

disabling anti-virus didnt help, saw the same error occur (and then reran it just fine)

[buzzqw]

@BlackDog21

wmv encoiding is quite a mess with wmvcmd.vbs.
i suggest to use the PS3 profile (x264) that will produce a mp4 file compatible with xbox360

@Encoder888

0.92b 01/12/2007
FIXED: Every profile that have in name "_CQ_" (without quotes but capitalized) will trigger the Constant Quality Window

but in AQ_Constant_Quality.xml is missing _CQ_ , so rename it to AQ_CQ_Constant_Quality.xml (fixed in next update)

However, from 0.91 on I've been able to do that with constant quality profiles
yes, the "specify quality" has priority versus CQ value

@Die's Man
check avs could bring to some weird behavior.. sorry.. de check it until future update :(

@spotter

when the audio isn't encoded, kill automkv
then start virtualdub and open mkvmaudio.avs
is now starting the encoding ?

BHH

[red5goahead]

I've some problem with DelayCut. I use Vista and last AutoMkv build. If I leave checked DelayCut the audio.mp4 file gone mute.
If I extract audio with dgindex work fine but audio processed by delaycut is mute. I tried with neroaac, lame or ac3 but same result. the problem is related to delaycut. do you have some tips about this?
edit: the source is a mpg or ts file ripped by sat. no problem on this source.

[delacroixp]

@Encoder888

0.92b 01/12/2007
FIXED: Every profile that have in name "_CQ_" (without quotes but capitalized) will trigger the Constant Quality Window

but in AQ_Constant_Quality.xml is missing _CQ_ , so rename it to AQ_CQ_Constant_Quality.xml (fixed in next update)

Nice one...a very neat solution !


:):devil::D
Pascal

[spotter]

@spotter

when the audio isn't encoded, kill automkv
then start virtualdub and open mkvmaudio.avs
is now starting the encoding ?

BHH

the audio is getting encoded, automkv for some reason thinks it isn't and then either deletes and rencodes (moving from nic source to directshow), or leaves it around and encodes an mp3 (when it moves onto ffmpeg/besweet)

and delaycut says the audio file is perfect

"Number of Errors= 0"

[buzzqw]

@red5goahead

mpeg/ts file from sat can be "wrong" in some way.. usually due to muxing

you can try fixing it with http://www.videohelp.com/tools/TS_Packet_Editor

OR in advanced settings check uncheck "Use dalay cut" and check "DSS for Audio"

@spotter
:mad: (not to you, but this problem is very very very strange and not replicable here!)

have you some source able to replicate ? could you post a link?

BHH

[spotter]

@spotter
:mad: (not to you, but this problem is very very very strange and not replicable here!)

have you some source able to replicate ? could you post a link?

BHH

I don't think its the source, I think it's my quad core machine.

Why would automkv assume the encode failed? What are the possibilities (i know if I hit "abort", it assumes it worked fine).

[buzzqw]

automkv will check for audio size.. and if less then 1000 bytes will throw the error

BUT the check is done after encoding!

the error is present for any audio codec selected ? for first and second audio track ?

try this version.. i added some prompt at audio encoding

http://www.64k.it/andres/data/a/AutoMKV093beta1-spotter.exe

BHH

[spotter]

it seems it can happen for any audio codec (haven't really tested, but all fallbacks have failed before)

any the file is defintely large enough, I see it growing during encoding.

trying the new version now to see if I can reproduce.

[spotter]

hmm, the new version when I load a vob tried to run dgindex right away instead of at encode time?

[spotter]

can't load via vob, it just hangs automkv, pegging a single core, seemed to load fine w/ an ifo

[spotter]

with ifo, no ac3 were extracted by dgindex. any other tests will have to be later.

[Die's Man]

@BlackDog21
@Die's Man
check avs could bring to some weird behavior.. sorry.. de check it until future update :(


Yeea, great, works again!

That's what I like, dealing with professionals :thanks:

[spotter]

with 0.91 I've closed every program that can be doing IO (I like to have nbpro open in the background doing its downloading thing) and haven't had an issue of yet.

It's not a space issue as I have 30-40GB free. I really think its just an IO stress issue, for some reason windows is returning a bad "stat" result if its done before the data gets a chance to "rest". One thing that might work would be to call the windows equivalent of a sync() syscall b4 you check the size. (I'm coming from the Unix world of programming, so unsure what the equiv in windows is).

[spotter]

In looking at the temp directory in explorer, it deffintely seems like a race of some sorts.

what I see

file growing in size
file drops to 0 size
file goes back to regular size

unsure what's causing it to drop to 0, but if that's the instant when automkv checks it, that's the problem.

[spotter]

further note, its full size while virtualdub is counting down to autoclose of the encode dialog, as soon as it disappears it seems to drop momentarily to 0 size, then back to normal size. guessing that if automkv gets in quick to check size, it see 0, and hence does a renecode which drops it to 0 again. When it works, it proceeds immeaditely to deint step

[weaver4]

I was wondering if someone could brief me on the "Specify Quality" percentage does? What range of values are good for xvid and x264? What profile(s) should I use with this option?

[buzzqw]

@spotter

thanks a lot for your test.. i will add a little delay before checking size.. and btw the linked exe is to be used with new dgindex . sorry :o

@weaver4

for x264 the crf is = 40-%quality_percent%/4

as profile i will use main profile

BHH

[spotter]

it's a guess the "delay" thing, for all I know I'm wrong, but if you have a beta for me to test (and a pointer to the new dgindex), I will try and test it. Though as my grandmother just passed away (and just got to my parents after a 4 hour train ride), I might not be able to test it so soon.

[buzzqw]

@spotter

download this http://neuron2.net/dgmpgdec/dgmpgdec150rc2.zip
unzip into \exe\dgindex and put dgdecode.dll to \exe\filter
update automkv with this build http://www.64k.it/andres/data/a/AutoMKV093beta2-spotter.exe

thanks

BHH

[delacroixp]

... as my grandmother just passed away (and just got to my parents after a 4 hour train ride), I might not be able to test it so soon.
Sorry to hear about your grandmother... which train did your parents take !


:):devil::D
Lacsap

[Mclfln]

buzzqw,
thanks for the help in your previous post. I have been experimenting with converting mpg files into AVI->XVID files. I have been setting the "Set Width Resolution:" to "original" but the output files have not been keeping the wide screen view of the original mpg/dvd rips. Any ideas?

Thanks again,
Matt

[buzzqw]

@Mclfln

try checking "Force Muxing AR" to 2.35

BHH

[BlackDog21]

@BlackDog21

wmv encoiding is quite a mess with wmvcmd.vbs.
i suggest to use the PS3 profile (x264) that will produce a mp4 file compatible with xbox360


I tried this and it resulted in a file that was around 8GB, over twice the size of a WMV. Is that expected?

Also, what settings should I use for Audio to ensure 5.1?

I used Original for Width and ran into the same issue as Mclfln, so I'll try your tip on Force Muxing AR.

[buzzqw]

I tried this and it resulted in a file that was around 8GB

ehmm.. what profile AND what Disk Settings ?

Also, what settings should I use for Audio to ensure 5.1?

select Nero and Original as downmixing

I used Original for Width
in this way the output file file will be 720*576 (or the ntsc value...)

BHH

[spotter]

so it doesn't seem the delay helps (though new automkv creates an aac file w/ ffmpeg instead of an mp3 like it did b4, so that's better, albiet at what seems a higher bitrate than nero does), I'm going to do some testing w/ just virtualdub when I get a chance to see if I can figure out whats going on when virtualdub exits.

[spotter]

change that, it deffinitely is jumping to 0 and back to regular size on virtualdubmod quit. Ran virtualdubmod w/o automkv and I saw it. Have no clue why this happens.

any chance you can give me a version that waits some long period (say 10 seconds) to check.

[buzzqw]

i will post tomorrow and update

thanks (and my condolence )

BHH

[BlackDog21]

ehmm.. what profile AND what Disk Settings ?


PS3_41 and "Don't care about size". I really don't care about size, I just want to get the best quality possible that streams to the xbox properly. If I can use a smaller size and still get that, great. Should I be using a different size setting?

[spotter]

in thinking about this more, I want to blame it on neroaac (well sort of, soundout doesn't let one work around it).

what's going on.

virtualdubmod is running an avisynth script, it calls the soundout() function

this function tells virtualdubmod to exec neroaac with the options to record from its stdin.

when soundout is finished, instead of closing stdout (and telling neroaac that its done) it just leave the "pipe" empty so neroaac blocks, waiting for input. At the end of the timeout, it closes and exits, and in the process, closes the pipe.

automkv sees that its closed, and tries to get the file size.

in the process, neroaac, sees the stdin was closed, does some funky stuff to the file (it seems to 0 it out and then recreate it, slightly larger, in explorer, its 1k, but that can be in reality a minimal difference).

virtualdub should not be exiting until either neroaac exits, or soundout should close stdout when its done, and then there wouldn't be a race condition (as the timeout for soundout, would take care of it in the latter case).

[buzzqw]

very very good analysis!

so the problem would be only in nero.. good to know

i build another test build

http://www.64k.it/andres/data/a/AutoMKV093beta3-spotter.exe

in this build you will get a messager after encoding AND after clicking ok, automkv will wait 7 seconds before checking for audio size

please report any problem

thanks again!

@BlackDog21

the key is "unlimited file size"
if you want quality but with a reasonable size just select PS3_41 profile AND in advanced settings check "Specify quality" to 80

BHH

[spotter]

ok, though that will make it harder to test throughly (i.e. was considering just giving it like 10 VOBs to encode and see if it ever failed its size test, with this, I'd have to babysit it a little, no biggy).

also posted the bug in avisynth dev forum,

http://forum.doom9.org/showthread.php?t=132912&goto=newpost

[buzzqw]

yea, saw, great bug post!

thanks again for all your effort!!!

BHH

[Mclfln]

Hi Buzzqw,

I tried this as well as 16:9 with the same results. Anything else I can try?

Thanks,
Matt


@Mclfln

try checking "Force Muxing AR" to 2.35

BHH

[spotter]

nothing's failed so far (though I tend to notice the dialog way after it shows up).

A better test to prove this point is for automkv to do a fake file test b4 it pops up the dialog (and print "would have failed, but will try after ok is pressed" if it detects a bad size).

[buzzqw]

new test build

http://www.64k.it/andres/data/a/AutoMKV093beta4-spotter.exe

in this build after virtualdub is closed automkv will wait 7 seconds, then will check for audio file.. if not found a message is prompted

thanks again!!!

BHH

[spotter]

ok, setup a queue of 17 encodes, will see if any fail.

[buzzqw]

:thanks: :)

BHH

[ryc0203]

I have seen many of the filters are compatible with mt
Even HybridFuPP 0.98 supports mt.
The question is AutoMkv is compatible with mt ? (using only filters compatible), and if this is how active the SetMTMode(2,0) before they run any other filter?

[spotter]

one "failed" in a weird way, it encoded a file of only 58k. Unsure why that happened (maybe I hit abort without realizing it), killed it when I realized that happened, restarted automkv (reloaded queue) and encoded file without a problem.

besides that, no problems w/ the "wait 7s" build, working w/o any babysitting.

[buzzqw]

@ryc0203

editing the base.avs should be a start.. on wiki there are even a simple approach.. look at it

@spotter

thanks for your test
about the broken's one.. something must appened during encoding... 58k isn't zero byte.. so .. a "hiccup" can be occurred.. :o .. don't know

just to know.. what kind of files have you queued ? have you got some problems with new dgindex ?

BHH

[survivant001]

@buzzqw

I my last test, I choose ConvertFPS for the audio.. and now I did a new test.. basic one.. just import a mkv. Choose AC3, original, 448 . that's it..

but AutoMKV remember the setting : TEMPO of the last session.. even in the textbox are at NONE.. in this test.

here the log

17:37:49 0.93 beta1 - - - - - - - - - - - - - START JOBS - - - - - - - - - - - - - -
17:37:49 OS Version: Windows Server 2003
17:37:49 Selected Input file: D:\DVD\mkv\the.victorias.secret.fashion.show.2007.720p.hdtv.x264-ctu.mkv
17:37:49 Selected Output dir: D:\DVD\converting\automkv\temp\
17:37:49 Selected Final Name: the.victorias.secret.fashion.show.2007.720p.hdtv.x264-ctu
17:37:49 Selected Container type: MKV
17:37:49 Selected Encoder: X264
17:37:49 Selected Size: 2 CD / 1
17:37:49 Selected Profile: 1_Pass_Fastest.xml
17:37:49 Selected Resizer: NONE / Original
17:37:49 Selected Filters: NONE
17:37:49 Selected Deinterlacer: NONE
17:37:49 Selected Field Order: AUTO
17:37:49 NeroAACEnc.exe: not present
17:37:49 Enabled CRF/CQ AutoRes at 18 value
17:37:49 Track Language 1: the.victorias.secret.fashion.show.2007.720p.hdtv.x264-ctu.mkv
17:37:49 Audio Codec 1: AC3
17:37:49 Audio Quality 1: 448
17:37:49 Audio Channel 1: Original
17:37:49 Track Language 2: NONE
17:37:49 Audio Codec 2: Lame
17:37:49 Advanced Audio Settings DUMP Track 1
17:37:49 Audio Track Language: und
17:37:49 Nero Option Profiles: AUTO --- Encoding Mode: Quality
17:37:49 CCT Option Profiles: AUTO --- Channel Options: AUTO
17:37:49 Lame Settings: ABR
17:37:49 Audio Normalization: 1
17:37:49 Frequency: 48000 --- Tempo: 24 to 23.976 --- Pitch: NONE --- Other Add:

[spotter]

I'm guessing you didn't hook up the delay for the 2nd "audio" track :)

[survivant001]

I'm guessing you didn't hook up the delay for the 2nd "audio" track :)

are you talking to me :)

if so.. I don't have audio track 2. I think automkv load the setting from the previous encoding, but didn't reset them in the current one.