BatchMux plugin - MuxMan authoring with DVD2SVCD [Archive] - Page 3

manolito

27th May 2007, 19:00

Edit. OK. here it is the link for a quick fix of the two problems reported above. Pulldown is now fetched from the command line (video file argument) and the presence of optional audio delay arguments should not break the plugin anymore.
For the moment, these arguments, if present, are just discarded - I will include them in the process in a future time...
Yes!!! Everything works as expected. Tried using CCE 2.70 with PAL sources (no pulldown) and with NTSC (with pulldown) both for DVD chapters and Fixed chapters, and the chapter points came out correctly every single time.

For the delay arguments I still try to figure out what DVD2SVCD is actually doing.

If the audio is reencoded, it seems that the delay correction is always done through BeSweet (using the -ota parameter) so the Mplex command line will not have delay parameters.

If "Keep source audio" is selected, I found two different scenarios so far. In one case the delay was -79ms, and this time BeSweet was run only to correct the delay. No delay parameter for Mplex. This was for a PAL source.

But in another case (NTSC source this time) the audio delay was 17ms, and this time BeSweet was not called at all. Instead there was the additional "-1 17ms" parameter for Mplex.

I would really like to know why the delay correction is sometimes done with BeSweet and another time with Mplex. Anyway, it would probably be a good idea to pass these delay parameters to MuxMan.

Cheers
manolito

Sir Didymus

28th May 2007, 10:15

Anyway, it would probably be a good idea to pass these delay parameters to MuxMan...

Yes! I fully agree with this. Beta4 should support audio delays...

Generally speaking, it can be not easy to understand what are the strategies applied by DVD2SVCD without the direct explainations of the authors. On the other hand, if the audio delay parameters are there, it's a good idea to support them.

An example is related to ac3 audio frames, that are encoding 32 ms of audio data, and are someway "atomic" units of ac3 audio. So it would be simply impossible for the audio encoder (e.g. BeSweet) to add or cut audio in order to compensate delays smaller than 32 ms. In this case the alignement must be performed at the mux level (by the authoring application)...

MuxMan is supporting audio delays in the range [-300..+300] ms. and BatchMux is already enabled to handle these parameters, so it turned out as easy to implement a simple check of the delays against the supported boundaries and to pass them to the authoring stage (dvdauth_hack.exe)...

EDIT. IMPORTANT... Please use the beta4 release of Mplex_hack below... Previous beta3 suffered from a nasty condition leading to loose the audio tracks for avi with audio delays... Beta4 should by OK, supporting (and delivering to the final DVD) also the audio delays... :)

Edit 20-06-2007 Link removed. See first page for the last available release...

BatchMux plugin - Released version 1.5
OK. It seems release 1.0 of Mplex_hack is working fine (no changes respect to Beta4 above). So I just update the first page of this thread with the most recent releases of the components:
Mplex_hack (ver. 1.0)
BatchMux (ver. 1.0)

Cheers,
SD

bionic

7th July 2007, 15:16

Hi guys! :)

Im testing avi to dvd, audio to 2ch ac3

Using all the latest versions incl aften rev 512.
Single core barton and cce 2.50 xpsp2

1. pal with ac3 5.1 = ok
2. pal with vbr mp3 = ok
3. ntsc 23.976 with vbr mp3 = ok
4. ntsc 23.976 with ac3 5.1 = ok
5. ntsc 23.976 with ac3 4ch(3.1) = got vobs only, doing another encode with batchmux only, no audio encoding.
Second round same result, this thing seem to pop up every now and again, im keeping the logs in case theres any interest in checking it out.

:)

Ive been trying to find a reason for the vobs only issue.
Afaict from the logs its the Muxman P-STD buffer underflow (bitrate spike?) issue thats been brought up from time to time.
Its right there in the logs, just took me some time to reflect.

Ive tried closed gops and that has sometimes worked, but very unreliable.
Ive tried lowering max bitrate and that works so far.
But theres no way afaik to predict what max bitrate values are needed without encoding several times.
And it sometimes takes lowering to an exctent that leaves the encode in cbr mode.

When lowering the bitrate enough for the authoring to finish "proper" i get the pulldown related 38 fields gop "error" in the muxman log
(1 oddities detected, resulting DVD is non-standard).
From what ive gathered so far this is related to encoding with 15 frames gops or more totalling to more than 36 frames after pulldown?

And since muxman counts the displayed frames/fields, not the physical frames in the stream. This has has an impact on max bitrate?
So could this problem be related to pulldown in some cases producing a too high max bitrate caused by the injected frames ?

Im still reading up on this though, input much appreciated :)

Boulder

8th July 2007, 08:46

Could someone please upload the latest version somewhere, sendspace.com seems to be down.

EDIT: Never mind, it's back up again..

Sir Didymus

8th July 2007, 19:43

Hi bionic!

See here for a nice discussion related to GOP sizes:
http://forum.doom9.org/showthread.php?t=113143

From what ive gathered so far this is related to encoding with 15 frames gops or more totalling to more than 36 frames after pulldown?

Yes, definitely.

And since muxman counts the displayed frames/fields, not the physical frames in the stream. This has has an impact on max bitrate?

No. The bitrate parameters (min, max, avg) are referring to the encoded frames. Applying pulldown to a given encoded stream should not change its bitrate...

So could this problem be related to pulldown in some cases producing a too high max bitrate caused by the injected frames ?
I can not see such a straight relationship. Maybe pulldown.exe is changing other timing information in the encoded stream (other than just adding the rff and tff flags), but I can not say it for sure, since I don't know the program...

Afraid I am not able to give you other hints on the matter...
It would be very interesting to create your assets using a different encoder, or to feed your assets to the professional release of MuxMan (the 0.18) to see if it makes any difference...

If you are able to make the issue replicable with short clips, maybe you may want to upload them somewhere (assuming no copyright issue holds), in order get them a look...

Please let us informed about your findings...

@Boulder: nice you are interested in the BatchMux Plugin :-)

Cheers,
SD

Boulder

8th July 2007, 19:48

@Boulder: nice you are interested in the BatchMux Plugin :-)I've set up DVD2SVCD for my dad since he captures a lot of TV stuff on his PVR-250..I just happened to browse through the forum and noticed that the brilliant multiplexer has been put to good use by you:) Too bad we're forced to switch to DVB at the end of August, DVD2SVCD won't work with DVB subs. I need to think of something for that (or would you like to hack D2S so that it accepts for example SUP files ;))

Sir Didymus

8th July 2007, 20:01

bionic

8th July 2007, 20:52

Hi Sir Didymus!

Thanks for the info and thanks for looking into it.
The link you pointed me to, along with this one
http://forum.doom9.org/showthread.php?t=116583&highlight=oddities+detected
was in fact the source of my "reasoning" :)

Yes, i do use muxman free version(0.15r) so testing with pro version would be something to try, i dont have it though, but i can up rather big clips for you to test with.

I was able to reproduce it with a test clip earlier, i dont have it anymore, but ill try to produce one that fails.

I have entertained the possibility of this being cce related, will test encode with quenc and tmpgenc too and report back.

I have a few logs with buffer underflow and the 38 gops error if you want to take a look?

Edit: Of course, if upping a clip is a copyright issue i will definitely refrain from it :)

Sir Didymus

8th July 2007, 21:21

I was able to reproduce it with a test clip earlier, i dont have it anymore, but ill try to produce one that fails.

I have entertained the possibility of this being cce related, will test encode with quenc and tmpgenc too and report back.

I have a few logs with buffer underflow and the 38 gops error if you want to take a look?

Of course, please post the logs... But I am dubtful it will be sufficient to understand what is going wrong... Most important would be to reproduce the issue and to have assets and logs available (in each step of the chain from the source asset, to the encoded one, to the pulldowned one)...

Boulder

9th July 2007, 03:35

I am quite interested in making the plugin as much useful as possible...

However I am nor sure I properly understand what you want to do... You mean you want to perform avi 2 dvd or dvb 2 dvd + subs in the sup format ? Have you some (short...) sample files to perform some tests ?

Edit - in any case it should be checked if other applications (e.g. FAVC ?) offer in a native way the features you are looking at... There is no point in developing some features that are already available elsewhere... By the way... FAVC is also using the jewel of Mpucoder (Muxman) for its authoring stage...Thank you for your interest. Unfortunately FAVC doesn't support MPEG2 input AFAIK so I can't set that up..

The problem is that the national broadcasting company (YLE) uses DVB subs so that they are not hardsubs but softsubs. When cutting the transport stream file with ProjectX, I can extract the subs directly to the SUP format. If D2S was hacked so that the SUP file could be imported when MuxMan does its magic, my dad could keep on using the program he likes to use and it would also save me from some headache.

I'll try to snip a sample for you tonight. ProjectX can extract to m2p or pva which D2S supports so that is not a problem. One problem is that to get the subs, you apparently need to demux the stream because when creating m2p or pva, ProjectX only adjusts the PIDs and keeps the subs in the container without extracting them as a separate file. But there is a solution to that (some manual work).

bionic

9th July 2007, 15:06

Quenc and tmpgenc tested.

Good news with Quenc.
1pass, default gops settings, high quality, trellis and extreme settings not active, muxman finsihes authoring proper.
No gops errors or pstd-underflow reported.

Tmpgenc tested twice. Just wanted to see if there was a difference.
One test with 2 pass vbr lowest quality rest at default.
One test with cq-vbr same settings.
Both tmpgenc tests produce several pstd-undeflow warnings in the logs.

Ill see about that clip next :)

Boulder

11th July 2007, 11:05

I'll try to snip a sample for you tonight.Dang, I forgot the sample:( Still, you pretty much don't need it, everything else can be done in the cutting phase in ProjectX before feeding the stream into D2S. The thing is to feed the subs in the multiplexing phase.

Sir Didymus

11th July 2007, 12:15

Quenc and tmpgenc tested.
Good news with Quenc...

Well, I am doing some testing using QuEnc and HCenc...

It seems to me (see advanced parametes of encoder) that using QuEnc for sources needing pulldown, the GOP size limit is set to 12. This explains why no GOP oddities are detected when using QuEnc as encoder.

For users who like adopting HCenc as encoder, I recommend for NTSC sources at 23,976 to adopt an HC configuration file (HC.ini), in the same place where HCenc.exe is located, containing the following line of text:

*AUTOGOP 12

It should solve the glitch of the GOP size limit with this encoder.

Edit. Thanks to the log files that bionic sent to me via PM, I discovered a bug in the chapter point generation performed by Mplex.hack. After some further investigation I found that the issue, under some non rare circumstances, may lead to the VOBS only problem reported above. In this scenario (affecting NTSC+pulldown cases) the error reported by MuxMan should be something like this:

Reference to non-existant scene "Seg1_Scn11" from PGC "VTS01_Ttn1_Pgc1"

The good news is that I was able to reproduce the problem, so I am positive to find a clean solution for this (in next days, most probably)... Stay tuned...

Sir Didymus

11th July 2007, 12:34

Dang, I forgot the sample:( Still, you pretty much don't need it, everything else can be done in the cutting phase in ProjectX before feeding the stream into D2S. The thing is to feed the subs in the multiplexing phase.

Don't mind...

But well, if some manual steps are still necessary in the process, then the simplest solution for adding a sup stream in the authoring stage is the following:

1) leave D2V making its job until the authoring phase, and then abort it.

2) You will see (in the DvdAuthor_log.txt) that a command line like this is issued:

BatchMux Arglist -->
-muxman "C:\Programfiler\DVD2SVCD\DVDAuthor"
-d "J:\Dvd2svcd\VIDEO_TS"
-v "J:\Dvd2svcd\Pulldown_Encoded_Video_NTSC.mpv"
-mxp "J:\Dvd2svcd\BatchMux.mxp"
-l "J:\Dvd2svcd\MuxMan.log"
-a1 "J:\Dvd2svcd\Encoded_audio_1.ac3"
-cellfr "J:\Dvd2svcd\CellTimes.txt"
-preparer "DVD2SVCD"
-a1lang --

3) in order to add a sup subpicture stream to the batch, it is simply necessary now to open a dos window and to paste/modify the command above in order to add the new subpicture (-s1 in the example):

BatchMux.exe
-muxman "C:\Programfiler\DVD2SVCD\DVDAuthor"
-d "J:\Dvd2svcd\VIDEO_TS"
-v "J:\Dvd2svcd\Pulldown_Encoded_Video_NTSC.mpv"
-mxp "J:\Dvd2svcd\BatchMux.mxp"
-l "J:\Dvd2svcd\MuxMan.log"
-a1 "J:\Dvd2svcd\Encoded_audio_1.ac3"
-s1 "New subpicture.sup"
-cellfr "J:\Dvd2svcd\CellTimes.txt"
-preparer "DVD2SVCD"
-a1lang --

Of course I should think to some way to make the step automatic, but in the meanwhile...

Another problem is that the colors and the display modes for the new stream are unknown a priori...

So, most probably some manual steps (PgcEdit editing) will be still necessary at the end...

If you launch BatchMux.exe manually, it will appear a short help showing the available options for the MuxMan wrapper; it is very simple to add or change some authoring parameter there...

Cheers,
SD

Boulder

11th July 2007, 12:45

Thanks, I think a simple batch file to run after each process might just do the trick, the sup filename just needs to be the same every time.

So you cannot enter the parameters for subtitle palette and display mode in the MuxMan CLI? Sorry, I'm at work and cannot try it out now myself.

Sir Didymus

11th July 2007, 15:22

Yes, thinking better at the problem, the solution of a simple batch file, at the end, should be the one solving your needs. You may enter a suitable palette, together with the other BatchMux parameters via the command:

-palette palette_file

It's an optional parameter; PgcEdit compatible ascii file containing 16 colour entries (index, red, green, blue) adopted for redefining the default color palette used by MuxMan.

There are also parameters for setting the subpic display mode... You just need to play a little bit with BatchMux... :rolleyes:

Boulder

11th July 2007, 15:58

OK, thanks - I'll play with BatchMux when I get the time :)

Sir Didymus

16th July 2007, 12:12

A new release of Mplex_hack is available:

edit - link removed. Please follow the discussion for the last available release

The code has been simplified, and the way of producing the chapters has been completely changed, in order to produce a chapters file for the authoring stage containing time references defined in terms of frame references or ND timecodes, depending on specific information available at the time mplex_hack is executed.

It should provide no improvements for PAL users, but some more precise results for NTSC users are expected (expecially in the avi 2 DVD mode and 24,976 fps sources).

Since the way of handling chapters is changed, please consider the release as a beta, and please report any oddities you discover...

Problems leading to a premature end of the MuxMan authoring, with some error condition similar to the following:

Reference to non-existant scene "Seg1_Scn11" from PGC "VTS01_Ttn1_Pgc1"

Should be fixed.

Cheers,
SD

bionic

16th July 2007, 12:22

Thanks :) testing as i post.

manolito

17th July 2007, 15:39

Test results for Mplex_hack.exe 1.1 beta:

I used two NTSC XviD Avi files, one @ 29.97 fps (no pulldown) and one @ 23.976 fps (pulldown). The clips both had a length of 2min 01sec. Fixed chapters every 20 sec were selected. CCE 2.70 was my encoder.

Correct chapter points in both cases. I then reverted back to the old Mplex_hack.exe 1.0, and indeed the chapter points created by the new version were slightly different.

Celltimes.txt created by ver 1.0:
600
1200
1800
2400
3000
Celltimes.txt created by ver 1.1 beta:
599
1198
1798
2397
2997

IMO these differences are not too relevant, for me it mostly shows that Sir Didymus is a real perfectionist...:)

One other thing I noticed: The clip has a length of 2min 01sec, but the last chapter at 2:00 is missing. This is a good thing, and it looks like this was done on purpose. Am I right?

Anyway, thanks for the new version, I could not find any issues with it.

Cheers
manolito

Sir Didymus

17th July 2007, 16:40

Test results for Mplex_hack.exe 1.1 beta:

IMO these differences are not too relevant...

agreed... :)

for me it mostly shows that Sir Didymus is a real perfectionist...:)

hem, agreed also with this one... :p Well, I have to explain that I implemented the small correction in the attempt to recover from a more serious situation (the authoring failures reported by bionic).

I have some evidence these errors are not related to the plugin, which seems working OK, apparently.

The point is that MuxMan seems quite sensitive to glitches in the authoring assets. Release 0.15R looks especially sensitive towards VBV underflows, refusing to complete the authoring stage in presence of such encoding errors. I have no clue on how to eliminate these errors. One recommendation is surely to try to limit the usage of the one pass VBR mode in CCE, which is disobeying to the max bitrate parameter... Another recommendation is not to push the max bitrate itself beyond 8500~9000 kbps... But I know both of the above hints are not eradicating the problem...

One other thing I noticed: The clip has a length of 2min 01sec, but the last chapter at 2:00 is missing. This is a good thing, and it looks like this was done on purpose. Am I right?

Yes, you are! There is a parameter (now it if fixed at 10 seconds or 250 frames, depending on different situations) preventing to generate a last chapter shorter than this parameter...

Thanks a lot for confirming Mplex_hack 1.1 is OK!

Cheers
SD

Edit - Spoke too early - sigh... Just found (thanks to bionic!) a little mistake in the new code introduced in mplex_hack beta1... Very nice it was discovered while mplex_hack 1.1 is in beta...
Please adopt beta2 here...

Edit 27-08-2007 Link removed - see first page of the thread for the most up to date version

ChickenMan

18th July 2007, 14:18

Ive been trying to find a reason for the vobs only issue.
Same here. I use to get the odd avi 2 dvd encode that didnt complete and reading the muxman log file, there were always references to buffer underflow probs. I sort of lived with that. But recently installing Nicks latest AC3enc8.exe and now only get the odd conversion thats complete and the rest not. So went back to AC3Enc7.bat and replaced the aften.exe with Aften 0.07 SSE3 optimized version and its now back to normal (ie. the odd conversion that never completes).

I've also noted of late, that on the odd occasion Muxman doesnt complete the authoring, the audio in the AVI was always odd (by odd, I mean either 44.1khz, 32khz or Mono). As a side note, Mono audio never seams to convert properly anyway, as Madplay produces a mono WAV correctly and then Aften (or the old AC3enc.dll) doesnt produce a compliant mono ac3 or a stereo ac3. Only 1/2 the audo is there but stretched out to full length of the movie. So audio way out of sync, very deep sounding and slow. I then manually process the audio.

Sir Didymus

18th July 2007, 16:37

Hi ChickenMan!

Thanks a lot for your comments, and for sharing your experience on the subject; very nice to know what you report about the audio assets :)

I am into these days more and more convinced that oddities in the assets feed to the authoring application are the specific cause of the reported issues.

In addition, please take into account that (in my experience) the glitches can be also related to the video alone.

We are having nice discussions and testing sessions (via PM) with bionic, where it seems - if confirmed - that under the specific conditions of the performed tests, the encoding based on CCE - multipass is getting rid of such errors, while OPV (and maybe CBR ?) is almost always producing buffer underflows at the authoring stage.

I have to say that I knew since some time that CCE - OPV is not complying strictly with the max bitrate encoding parameter... This single finding is quite astonishing and difficult to believe, considering the level of such a professional product... But there is some clear evidence about this...

And the implications are really serious, considering that multipass encoding takes much longer than OPV, and that the implicit suggestion (to not use OPV) is quite embarassing, looking for example at the huge - and beautiful - work performed by Tylo on the Roba method...

However it has been not easy to establish that the BatchMux plugin (and MuxMan, of course) have no responsibilities on the issue...

After all (I hope you agree), this is a good reason for going on using BatchMux:

better to know your assets have some troubles and stop the authoring, than producing a DVD with glitches and realising it stutters only at the playback time... :p

Cheers,
SD

manolito

18th July 2007, 17:24

AFAIK Mono audio has always been a problem. I recall some older posts about it (I think they were by Nick), and I made it a habit a long time ago to convert Mono to Stereo before feeding the audio to Besweet.

But I never had a problem so far when source audio was 44,1 kHz and had to be converted to 48 kHz.

As for the muxing failures caused by buffer underflows: Muxman is extremely picky and does not finish the conversion if it thinks that anything about the source files is non-standard. But on the other hand this means that if Muxman does not complain then you can be 100% sure that your encode is DVD compliant (even if there is an occasional bitrate spike above the max bitrate limit).

CCE OPV is my standard conversion method, and yes it does overshoot the specified max bitrate here and there. My max bitrate is always set to 8000 (DVD2SVCD sets this as the default), and using this value for max bitrate makes my encodes "Muxman proof" in 99.9% of cases.

QuEnc 1-pass VBR is a different story. Even with max bitrate set to 8000 there are buffer problems quite frequently, especially with short encodes. A way to get rid of the problems usually is to lower max bitrate even more and to use a high bitrate matrix like Fox Home Entertainment (or to switch to CBR).

So IMHO all this has nothing to do with Batchmux as Sir Didymus already pointed out. In the rare cases when Muxman refuses to convert my source files I usually try to remux with Imago and reauthor with DVDAuthor. Even if DVDAuthor issues some warnings, there is a very good chance that my standalone will play the DVD without any problems.

Cheers
manolito

bionic

18th July 2007, 18:34

Ill throw in a cce 2.50 opv encoding with max birate 8000 too then :)
I usually use 9000 or 9200 as max, but ive had assets that required iirc no more than 5000 as max for authoring to finish proper with muxman.

ChickenMan

20th July 2007, 14:32

I also use CCE 2.50 OPV 99% of the time ( to many hassles with 2.70 in Batch mode) but I also got the odd non-completed VIDEO_TS folder before SD's Muxman came along. That was using standard DVDAuthor, so I dont believe the problem lies with Muxman. But with all the new Aften versions of late and Muxman with various combinations of both seamed to make things worse, not better. But at the moment, 90% of encodes are now coming out fully authored. By the way, I have always used a max bitrate of 9200 for the last few years.

Those avi's with 44.1 audio, well the audio is converted to ac3 at 48kh fine, just it almost always doesnt get authored complete.

manolito

21st July 2007, 12:48

By the way, I have always used a max bitrate of 9200 for the last few years.

Here is the Muxman log file from an OPV conversion I just finished. I used CCE 2.70 with D2SRoba. The encode was done with a Q of 7, max bitrate set to 8000, min bitrate was 1000. No filtering at all, Quantizer characteristics 28, DC precision 10, GOP size 12, standard matrix.

MuxMan version 0.15R
Accepted video I:\Movies\Encoded_Video_CCE_PAL.mpv size = 4127779160
no multichannel extension found.
Accepted audio I:\Movies\Extracted_audio_1.mpa

13:09:31 Begin multiplex VTS01.
Title Segment List
Segment_1
Maximum audio duration 391558 fields.
Starting scene Segment_1_scn1 at 00:00:00:00
SeqEnd at F608E954.
Bytes remaining in buffer = 0.
Bitrate - avg: 4510552, min: 921600 (lba 2128), max: 10308266 (lba 2087389).
Shortest GOP has 12 fields, longest GOP has 24 fields.
Fields: 391556, VOBU: 15762, Sectors: 2155925.

13:27:00 Begin multiplex VMG.
13:27:00 End multiplex.

As you can see, Muxman reports a max bitrate of 10308266 which is definitely higher than the DVD specs allow. But still Muxman had no problem authoring the files correctly, and there is no stuttering on my standalone whatsoever.

So I think you can make your life a lot easier if you limit your max bitrate for OPV encodes to a value of 8000. I do not believe that your quality will suffer in any visible way by this bitrate restriction.

Cheers
manolito

ChickenMan

22nd July 2007, 01:46

Thanks Manolito, I iwll drop it back and see how it goes. I also have seen the log files saying Max bitrate way over dvd specs and both authored completely and non-authored. Maybe because I've had the max bitrate so high at 9200 and audio at 224 leave little elbow room for either to go over a bit as why I have found the actual version of aften to have influenced whether it authors complete or not.

Back to 8000 and will report back. But I will keep my Min bitrate at 200.

manolito

8th September 2007, 18:15

@ Sir Didymus

I just noticed that for version 1.6 you changed the name for the chapter points from "Celltimes.txt" to "CellTcodes.txt". Does this change apply in all cases, or are there situations where the old name is used?

The reason I am asking is that my little batch file for automatic PALtoNTSC conversions ( http://forum.doom9.org/showthread.php?p=992798#post992798 )has to work with this text file in order to adapt the chapter points to the new frame rate. If the new file name is always used I just have to change two instances of the file name. If not then I will have to add a routine to parse the log files to retrieve the correct file name.

Cheers
manolito

Sir Didymus

8th September 2007, 20:45

Hi manolito!

Edit. I completely edited out my previous description (of 1 hour ago). It contained just plain wrong information...

The process in Mplex_hack is as follow:
1. The program looks in the file "dvd2svcd project file.d2s" for the movie length (in hh:mm:ss)
2. If this item is not zero, then the movie length is used, and the chapters are generated in NDTC format (CellTcodes file)
3. otherwise movie length is recovered from the ecl file, and the chapters are generated in frame format (Celltimes file)

The native format for MuxMan (scene points in the mxp files) is based on the NDTC format - i.e. "hh:mm:ss:ff" strings, so I decided to change the previous scheme, in order to avoid a first conversion of scene points from NDTC to movie frames (in Mplex_hack) and another conversion (in BatchMux.exe) from movie frames back to NDTC scene points... I am quite convinced this way of handling available information is the most linear and precise...

I want to keep interoperability with your script. It seems to me very useful. Sorry if I broke it. Let's think now about how to recover... :)

Edit2. I just got a deep reading of your script. I see the problem: it assumes CellTcodes file is still in a frame based format... It could not work this way. I am really afraid the last release of mplex_hack caused this trouble. :mad:

Am I right stating the the activities performed by your bat script are as follow ?
a) to change all the chapter points;
b) to apply DgPulldown to the source.

dirio49

9th September 2007, 00:39

@Sir Didymus

Is there any way you can make BatchMux to also work with titlewritter?

I know that you intent BatchMux to work with dvd2svcd.
but Titlewriter uses the same utilities for what it does.

Thanks
later

Sir Didymus

9th September 2007, 11:14

Is there any way you can make BatchMux to also work with titlewritter?

Sorry I don't know this application. I will try to learn something more about... The basic purpose (and functionnalities) of BatchMux are to provide a rich command line interface to the authoring engine of MuxMan. I don't even know is what you are proposing is meaninful or not... What does titlewriter do ? Is its author active ?

I know that you intent BatchMux to work with dvd2svcd.

That's only partially true. The authoring layer provided by BatchMux is also used by FAVC. For the future I am planning to get in contact with authors of other one click applications that have been recently made available, in order to understand if it may exist some valid reason in order to use an additional, alternative, authoring engine respect to the one these adopt so far... From my point of view the ideal situation is to be able to choose among different authoring possibilities... For DVD2SVCD this goal is (more or less manually) achieved...
:)

dirio49

9th September 2007, 14:33

Thanks for the reply.
What does titlewriter do ?

Titlewriter is a program used to Add/Edit DVD_TEXT, Menus (Either Simple or Original) into Dvd Compilations (Menu Area has been heavily designed to work with DVDShrink Reauthored Compilations).

Is its author active ?
yes, FallenAngel (http://tinyurl.com/2xhpmc)
hi is active in Digital digest most of the time

manolito

9th September 2007, 19:16

@Sir Didymus

I did some testing this afternoon, and the results are a little different from what you wrote:

The process in Mplex_hack is as follow:
1. The program looks in the file "dvd2svcd project file.d2s" for the movie length (in hh:mm:ss)
2. If this item is not zero, then the movie length is used, and the chapters are generated in NDTC format (CellTcodes file)
3. otherwise movie length is recovered from the ecl file, and the chapters are generated in frame format (Celltimes file)
In DVD2DVD mode, fixed chapters, QuEnc encoder the D2S project file does contain the correct movie length (in hh:mm:ss). According to your post Mplex_Hack.exe should create a chapter file in NDTC format in this case, but it doesn't. It creates a "Celltimes.txt" file in the frame index format instead.

Generally in DVD2DVD mode I found that the chapter file will always be in frame index format, no matter if I used fixed chapters or DVD chapters, or if I used QuEnc or CCE 2.70. Only in AVI2DVD mode using QuEnc I did get a chapter file in the NDTC format.

The native format for MuxMan (scene points in the mxp files) is based on the NDTC format - i.e. "hh:mm:ss:ff" strings, so I decided to change the previous scheme, in order to avoid a first conversion of scene points from NDTC to movie frames (in Mplex_hack) and another conversion (in BatchMux.exe) from movie frames back to NDTC scene points... I am quite convinced this way of handling available information is the most linear and precise...
I am not so sure. The NDTC format may be the native format when you use Muxman in command line mode. But in GUI mode Muxman only accepts chapter files which are in the frame index format.

Am I right stating the the activities performed by your bat script are as follow ?
a) to change all the chapter points;
b) to apply DgPulldown to the source.
Yes, that's the basic idea. Additionally the script detects if the Avisynth script contains a frame rate conversion to 59.94fps in which case the DGPulldown step is skipped. But in both cases the PAL chapter points have to be recalculated for the new presentation frame rate.

Anyway this PALtoNTSC script is not the main reason why I do not like the new chapter file format. I just wrote this batch file because first of all there is a real need to do this conversion (many NTSC standalone players refuse to play PAL DVDs), and second because the "Celltimes.txt" chapter format made it very easy to convert the chapter points using just a simple batch file. Converting chapter points in the NDTC format is a lot more complicated, at least for a batch file.

I frequently interrupt the DVD2SVCD conversion to do some tweaking. Sometimes I want to sweeten the audio track in WaveLab, and then I manually call Muxman to get my final DVD files. So far I was able to use the Celltimes.txt file which had been created by Mplex_hack.exe for this final authoring. With the new chapter format I will have to create the chapter file from scratch.

Right now I think that I will revert back to Mplex_hack.exe 1.0 from the Batchmux 1.5 package. I will gladly sacrifice a little accuracy of the chapter points in exchange for the convenience of having to deal with only one file name and one format for the chapter file.

Cheers
manolito

Sir Didymus

10th September 2007, 19:30

Hi manolito! :)

...In DVD2DVD mode...

You are right, I posted just the situation related to the avi2dvd case, since it seemed the one we were discussing about... In DVD2DVD mode the chapters are always already available...

I am not so sure. The NDTC format may be the native format when you use Muxman in command line mode. But in GUI mode Muxman only accepts chapter files which are in the frame index format.

Try to load a video file and whatever Celltimes file in Muxman, then save the project and open it with a text editor. You will see the native format, which is in NDTC...

Yes, that's the basic idea. Additionally the script detects if the Avisynth script contains a frame rate conversion to 59.94fps in which case the DGPulldown step is skipped. But in both cases the PAL chapter points have to be recalculated for the new presentation frame rate.

I asked because I was thinking at the possibility to incorporate inside Mplex_hack the process performed by the pal2ntsc script, but now I have a better idea...

...Converting chapter points in the NDTC format is a lot more complicated, at least for a batch file.

That's understood.

Right now I think that I will revert back to Mplex_hack.exe 1.0 from the Batchmux 1.5 package. I will gladly sacrifice a little accuracy of the chapter points in exchange for the convenience of having to deal with only one file name and one format for the chapter file.

I will try to avoid this need to roll back to a previous version of the plugin. Could it be acceptable, when the NDTC files are generated, to have also the Celltimes.txt produced ? The authoring would be normally based on the NDTC files, but at least you have available the other format in case of need (manual processing and pal2ntsc conversions). What do you think about ?

Cheers,
SD

manolito

10th September 2007, 22:38

I will try to avoid this need to roll back to a previous version of the plugin. Could it be acceptable, when the NDTC files are generated, to have also the Celltimes.txt produced ? The authoring would be normally based on the NDTC files, but at least you have available the other format in case of need (manual processing and pal2ntsc conversions). What do you think about ?
This would be very nice. But how would you assure that if a PAL2NTSC conversion took place the right chapter file is used for authoring?

My batch file would only correct the chapter points for Celltimes.txt while CellTcodes.txt would still contain the uncorrected chapter points. Somehow DVDAut_hack.exe needs to know which of the two files it should use.

I already started to explore if I somehow could manage to correct chapter points in NDTC format using a batch file. WinXP batch files are quite powerful (Nick tought me some tricks), but so far I have not come up with anything useful.

Cheers
manolito

manolito

11th September 2007, 00:08

Well, I think I gathered enough information now so I am confident that I can come up with a new PAL2NTSC script which can handle both chapter formats. Just give me a couple of days...

Cheers
manolito

Sir Didymus

11th September 2007, 07:56

Just give me a couple of days...

It was the same I wanted to say... :cool:

Yesterday evening I investigated further on the way PgcEdit and MuxMan handle chapter point and timecodes. For NTSC sources it is always applied the conversion factor 1 second --> 30 frames. So (for alignement and compatibility reasons) I decided to change back again the way Mplex_hack is working. It seems that converting 1 second to 29,97 frames is not correct...

Sorry for these indecisions concerning NTSC. Living in PAL land, my neurons run at 25 fps...

OK. In short, I need to produce a new release of Mplex_hack, so please wait a little bit, since I also found a coding solution which is clean and general, in order to obtain the desidered precision using the Celltimes format alone...

Within a couple of days...

Maybe even tomorrow (the changes respect to the beta release below are quite little, but in these days I can work on BatchMux just in the evenings: here at work I am changing PC, so the C compiler and some other development facilitis are temporarily unavailable to me)...
:)

Cheers,
SD

Edit. This beta1 release of Mplex_hack 1.2 is producing always both formats...
http://www.sendspace.com/file/7qdqzp

manolito

11th September 2007, 19:59

Hi SD,

the problem how to convert chapter points in NDTC format is solved - it simply is no problem at all. In NDTC format the chapter points are absolute time references, and the frame rate of the movie is totally irrelevant. No need to convert anything. The chapter points will be correct no matter if the movie is PAL or NTSC. How could I be that stupid! :confused:

This means that using Mplex_hack.exe version 1.1 all I would have to do to adapt my PAL2NTSC script is to check if Celltimes.txt does exist. Your new Beta of Mplex_hack 1.2 requires no action at all.

But during testing I discovered another problem which has been there for a while:

Using CCE 2.70 in AVI2DVD mode with fixed chapters Mplex_hack.exe has to use the ECL file to get the number of frames and the D2S project file to get the frame rate, because the movie length in the project file is 00:00:00. This works as long as there is no frame rate conversion during encoding.

One of my methods to do the PAL2NTSC conversion is to use "ChangeFPS(59.94)" in the Avisynth script. In this case Mplex_hack.exe detects the movie length incorrectly. The detected length is too high, so some extra chapters beyond the length of the movie are created which causes Muxman to fail.

I do not know how you get the frame rate in this case, but to me it looks like you assume that if PAL=1 then FPS=25. This does not work here. The D2S project file still says "PAL", but the "FPS=" entry does show the correct frame rate of 29.97 fps. If you use the "FPS=" entry you will get the correct movie length. In my tests the detected length looked like you had used a value of 25 for the frame rate.

OK. In short, I need to produce a new release of Mplex_hack, so please wait a little bit, since I also found a coding solution which is clean and general, in order to obtain the desidered precision using the Celltimes format alone...
I believe there is no need for this. The CellTcodes format is just fine. If you could just check the routine for movie length detection...

Cheers
manolito

Sir Didymus

12th September 2007, 08:24

The chapter points will be correct no matter if the movie is PAL or NTSC...

Ahhh, right!.... Silly me I did not point out before this very strong consideration!

But during testing I discovered another problem which has been there for a while...

Oh, Oh.... Your point is understood... I need to change the segment of code looking at the PAL flag and introduce a new check involving FPS...

In any case I think it would be much better to use a single format for the chapters exported by Mplex_hack. I can actually use both, but I would prefere making a choice. The generality of the NDTC format is appealing, and the involved simplifications for external processes (avoiding to change the chapter file within pal2ntsc conversions) would bring me on the NDTC side: maybe using this format always, in all situations, is the definitive solution we are looking at...

What do you think about ?

manolito

12th September 2007, 12:56

The generality of the NDTC format is appealing, and the involved simplifications for external processes (avoiding to change the chapter file within pal2ntsc conversions) would bring me on the NDTC side: maybe using this format always, in all situations, is the definitive solution we are looking at...
Yes, that would be nice. Maybe you could still create the additional Celltimes.txt in parallel (without using it). This would make manually authoring later using Muxman a lot easier...

Cheers
manolito

Sir Didymus

12th September 2007, 13:54

Maybe you could still create the additional Celltimes.txt in parallel (without using it). This would make manually authoring later using Muxman a lot easier...

That's right, but it would be not a good solution from the point of view of the interoperability of the plugin with other addons like your script. If both files are created by Mplex_hack, then it should be assumed that they represent the same chapter points. This condition can not be maintained if pal2ntsc (or other scripts and/or avisynth filters) applies pulldown and/or change TV system and/or framerate to the video. After one of the above step, the two formats will represent different chapter points.

I am quite convinced that a single output format from Mplex_hack is the best solution. I am just not sure about which format to select:

NDTC. Pros: It is an absolute representation of chapter points in terms of hh:mm:ss:ff time strings. It is independent from the TV system (PAL, NTSC) and FPS. Cons: in case of manual processes (e.g. loading of chapters in the MuxMan GUI - free version) the format is not suitable, and a new Celltimes.txt file should be manually generated.

Celltimes format. Pros: Very popular and compatible with most of the applications all around, including the MuxMan GUI. Cons: among the other, if the TV system changes (pal2ntsc conversions) the file should be changed and a new Celltimes.txt file should be generated.

manolito

12th September 2007, 14:45

Either one of the formats is fine for me. If you decide to go NDTC I will probably write a small tool which converts CellTcodes.txt to Celltimes.txt. Just specify the input file and the frame rate, and the corresponding Celltimes.txt file will be created.

Cheers
manolito

Sir Didymus

13th September 2007, 09:59

Well. I confirm I am keeping a single output format. The following beta release:

Edit: link removed. Please look at the first page of the thread for the most recent release...

Is based on Celltimes.txt format (movie frames). It now checks also the framerate, in a very simple way, as described below. I did some limited testing, but I am not sure about interlaced PAL sources. Is in this case the fps set to 25 or to 50 ?

There are also other possible sources of troubles related to the fact that the Pulldown flag is simply derived from the argument of Mplex_hack, and in this case, always the MovieFrames, when used, are scaled by the fixed factor of 5 / 4.

The overall process performed by mplex_hack to generate the Celltimes.txt file, is shortly summarised as follow:

Overall process for recovering movie length & chapter points, looking in the "dvd2svcd project file.d2s" file:

1. DVD2DVD mode.
- Framerate (initialised to 30 fps)
- TVsystem (acquired from "PAL=" string)
- MovieFrames (acquired from "Frames=" string)
- If the line "fps=25" is present in the file, then FrameRate is set to 25 fps
- If the line "fps=" is present in the file, then FrameRate is set depending on the TV System (25 for PAL and 30 for NTSC)
- In case MovieFrames is 0, then Mplex_hack attempts to get this value from the CCE ecl file
- In case Pulldown is present, as detected from the argument of Mplex_hack, then MovieFrames = MovieFrames * 5 / 4

2. Avi2DVD mode.
- Framerate (initialised to 30 fps)
- TVsystem (acquired from "VideoFormat=" string)
- MovieLength (in hh:mm:ss format, acquired from "Length=" string)
- If the line "fps=25" is present in the file, then FrameRate is set to 25 fps
- If the line "fps=" is present in the file, then FrameRate is set depending on the TV System (25 for PAL and 30 for NTSC)
- In case hh, mm and ss are all 0, then Mplex_hack attempts to get MovieFrames from the CCE ecl file
- In this case, when Pulldown is present, as detected from the argument of Mplex_hack, then MovieFrames = MovieFrames * 5 / 4

Then,

a. DVD chapters.
- The file "dvd2svcd chapters file.ini" is expected to contain valid chapter points,
that are simply extracted from the file, and scaled by the factor 5/4 in case Pulldown is present

b. fixed chapters.
- Both in case of Movie Length is based on hh:mm:ss and on MovieFrames, the Celltimes.txt file
is generated based on chapter points = chapter lenght * i * Framerate.

manolito

13th September 2007, 15:15

Thanks SD for the new Beta. I have already started testing, no problems so far. I will post the results once I have finished all tests (probably later tonight), but I already want to let you know about two slight peculiarities:

When I switched between QuEnc and CCE 2.70 I noticed that with CCE one more chapter at the end of the movie was created. My source was a 2 min AVI with fixed chapters every 10 sec, and the chapter @ 01:50 was not there when I used QuEnc. Simple reason is that with CCE one more frame is reported as the No. of frames. So you probably should not increment the value for the last encoded frame from the ECL file.

The other thing has nothing to do with Mplex_hack, it is a DVD2SVCD issue. For my tests I first used an AviSynth script where I had commented out all commands. The source already had the correct size, so no resizing was required, and I did not want any filters. The encode using QuEnc went well, but no chapters were created. The reason was that D2S reported only 240 frames and a length of 10 sec for the source. All I had to do to make D2S report the correct length was to uncomment just one command in the script (I uncommented the resize command which would not be executed anyway because Avisynth is intelligent enough to not invoke the resizer if the source format is already correct).

Cheers
manolito

Sir Didymus

13th September 2007, 17:44

Simple reason is that with CCE one more frame is reported as the No. of frames. So you probably should not increment the value for the last encoded frame from the ECL file.

That's good to know and easy to iron out :thanks:
Did you check, after the encode, that the length of the encoded movie is equal to the source, and still one frame less than what reported in the Mplex_hack logs?

The other thing has nothing to do with Mplex_hack, it is a DVD2SVCD issue...

That's good to know too... Whatever suggestion in order to recover also from this very special condition is welcome...

Cheers,
SD

manolito

13th September 2007, 22:50

Did you check, after the encode, that the length of the encoded movie is equal to the source, and still one frame less than what reported in the Mplex_hack logs?
To be honest, I did not even check. One frame more or less, who cares? It was pure coincidence that with my 10 sec fixed chapters I hit Mplex_hack's threshold where it decides if the last chapter should b created or not. Who in the world uses 10 sec chapters? I just wanted to point out that I detected a difference between QuEnc and CCE, but I think it is completely irrelevant if you take the time to change your code for CCE or not. Noone will ever notice...

That's good to know too... Whatever suggestion in order to recover also from this very special condition is welcome...
Another thing you should not even bother. Since the source code of DVD2SVCD is not available, nobody can tell how D2S calculates the movie length which is visible in its project file. I believe that even the author of D2S did not trust this calculation very much, because if you use D2s with the default Mplex / DVDAuthor combination, this movie length is not even used. Right after muxing there is some other routine called "Determining Length of Audio" which is used for chapter creation for DVDAuthor. When using the Batchmux plugin this routine always returns a vale of zero, so it is useless here.

But here are my test results for Mplex_hack.exe 1.2 Beta2:

In short, I could not find any issues. My tests were quite comprehensive, I had to create my NTSC sources from scratch, and I tested almost any possible combination. No problems whatsoever...

I did some limited testing, but I am not sure about interlaced PAL sources. Is in this case the fps set to 25 or to 50 ?
FPS is always 25, no problems here.

The only thing I have to change is my PAL2NTSC script. My script supports two different methods for the conversion. The first method uses DGPulldown. No changes are required here. MPlex_hack.exe cannot detect the (irregular) pulldown applied by DGPulldown, so my script has to convert the chapter points.

The second method works by converting the frame rate to 59.94 fps, and up to now this conversion was not detected by Mplex_hack.exe, so my script would have to do this. This has changed with the current Beta. The new frame rate is detected, and my script has nothing to do here.

It took less than a minute to update my script, so I am very happy with this current version.

Cheers
manolito