GPL command line M2V transcoder

[jhmac]

@Nic & other ReJig developers

I would like to make a donation (PayPal) to the ReJig cause, do you have something setup yet? A little something to say thanks for all the hard work...

[Nic]

No need for donations. Give it to a local cause or charity :) Thanks alot for the nice offer though :)

-Nic

[DSPguru]

Originally posted by Nic
Give it to a local cause or charity :)Nic is dA'mAn!

[mmgrover]

See, Nic doesn't need it, Him and bill gates live on the same street!

Just kiddin, lol
mike

On another note, Nic what are your plans for rate control?

mike

[E-Male]

is dvdauthor only for movie-only, or can it create complete DVDs?

EDIT: afetr readign a bit more, seems like it is (or will very soon be) able to do whole DVDs

So if the code of rejig, DIF4U and dvdauthor could be joined to one program, we could in the near future be able to do full dvd-backups with freeware

[jdobbs]

Once again my hat is off to Nic. You're an inspiration.

[onesoul]

Originally posted by Nic
Excellent news (Thanks for pointing this out to me Int21h). I'm crazy busy right now with work, but this deserves my attention :) Ill try and port M2VDownsizer now.

-Nic

ps
Edit: Well, already got it compiling, now just got to get all the 24 unresolved errors removed. I haven't put in the MMX code or anything yet, expect the first release to be slow as hell.
Edit2: Although it might not help a jot, i did build a ICL 7.1 compiled Requant http://nic.dnsalias.com/Requant.zip
Edit3: Got it compiled as an exe, now gotta work out how to use it ;)
Edit4: Uggh, it's one of these apps that uses stdin and stdout for the input/output...that's just wrong, im on windows afterall ;) soon change that.
Edit5: Well it's running, using 99% cpu so I suppose it's going ok. If I can't get it working then Ill at least release the source code, got to get back to work now. It's all done in VC6 using ICL 7.1.
Edit6: Well now its running correcly ;) Very very slowly making output...really need to get the mmx code in, may not put it out till then.
Edit7: Wow! It works...did my first little test. Slow as hell though.
Edit8: By the way, ill make a little GUI for Requant and make it so it can accept PS/TS/ES Streams as well (although it will only output ES)

Sorry for going back, but I didn't understand what M2Vdownsizer can do. Is it better than M2VRequantizer and usable?

cheers

[mmgrover]

Can someone offer the ported version of M2VDownsizer?
I would like to do some Quality tests :-)

mike

[int 21h]

Originally posted by onesoul
Sorry for going back, but I didn't understand what M2Vdownsizer can do. Is it better than M2VRequantizer and usable?

cheers

M2VDownsizer is a full decoder/encoder implementation, based upon Mpeg2Enc if I remember right... its very slow and not very usable, the quality is ok though.

[mmgrover]

@int21h

Maybe M2Vdownsizer isn't worh messing with ...

[int 21h]

Nic has a completely ported version of it... in fact before Requant came out, I think he was working a Mpeg-2 encoding project with it... In any event, as far as I know, Nic looked at it, and even added some of the optimized routines back in, and it was still pretty slow, and so it was put on the very, very back burner for now.

[jsoto]

a) When "Correct audio delay" is selected, even streams with 0ms delay are "corrected".

This could be a good practice. No delay correction, but some common errors in ac3 (crc, uncompleted frames, etc) could be fixed or, at least, detected.

jsoto.

[Exo]

Isn't it possible to requantize with another matrix?
This could possibly increase quality when using higher
compression ratios. CCE (newer than 2.50) for example
lets you choose between different matrices. Two of
them are called: very low bitrate matrix and ultra
low bitrate matrix. Maybe this could lead to higher
compression with less artefacts?

[jhmac]

Sure got quiet around here...

[int 21h]

Originally posted by Exo
Isn't it possible to requantize with another matrix?
This could possibly increase quality when using higher
compression ratios. CCE (newer than 2.50) for example
lets you choose between different matrices. Two of
them are called: very low bitrate matrix and ultra
low bitrate matrix. Maybe this could lead to higher
compression with less artefacts?

I was going to reply to this earlier, but I've been drunk most of the weekend... but here I am yet again :)

A quantization matrix is used during encoding to specify levels of quantization for specific frequencies (i.e. usually you apply a high level to things not seen very often, and a lower level to components that make up more of the picture)

When you look at a compression matrix, for instance the KVCD compression matrix...

Intra:
8 9 12 22 26 27 29 34
9 10 14 26 27 29 34 37
12 14 18 27 29 34 37 38
22 26 27 31 36 37 38 40
26 27 29 36 39 38 40 48
27 29 34 37 38 40 48 58
29 34 37 38 40 48 58 69
34 37 38 40 48 58 69 79


You'll notice we have 64 integers listed (these are used in the quantization equation as a sort of constant), these correspond to the 64 coefficients that the DCT process yields. Due to the nature of DCT, the block itself becomes arranged by frequency, with the 0,0 corner (the DC) holding the most significant information, and the 63,63 (one of the ACs) corner holding the most insignificant information, so that after you apply quantization to a block, you get something like this:


153 5 4 0 0 0 0 0
4 2 0 0 0 0 0 0
3 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0


And something like this is what is stored (all the 0's aren't actually written out, because in conjunction with zigzag scanning and the EndOfBlock symbol, you don't need them..)

So.... In ReJig, using a re-quantization matrix would probably not be the most direct approach to a quality based encoding model. Something like a matrix might be ok, but we really don't need to specify requantization parameters for all 64 coefficients, because its probably very rare that they all even exist. The artifacting problem is happening when extra quantization is being applied to a frame, and its reducing the ACs of the frame (the coefficients other than 0,0) to 0 or near 0.

This all really leads up to the problem that ReQuant currently is a unintelligent engine. Right now it blindly applies the rules it has to every frame, no matter what is in the frame or what the result is... I imagine this will be improved in time, just be patient :)

[Exo]

@ int 21h:
thanks a lot for your detailed explanation

Originally posted by int 21h
just be patient :)

Well, I'll try :)

[robshot]

Posting to inform some strange behaviour.

1. Try to backup ALL content of Ride with the Devil R1
2. using rejig to "jig" movie content in VTS_06_1.VOB to VTS_06_06.VOB.
3. result is fine, big .m2v file.
4. use ifoedit 0.92 to open the VTS_06_0.IFO (whereas VTS_06_0.VOB is the menu).
5. under ifoedit, i go to VOB Extra, and remux the new .m2v. Note: UNCHECK the AutoCopy Menu Files to Destination, and VOB size is 1GB.
6. This DVD has trailers and music videos by Jewel, which I also rejig because i want full BACKUP. Trailers+Music Video in VTS_03_1.VOB, and Recommendation Trailers in VTS_01_1.VOB
7. I took the same procedures with the movie rejigging (Opening original VTS_01_0.IFO, then opening VTS_03_0.IFO)
8. remux with VOB Extra menu of ifoedit 0.92, the same process with the movie.
9. I then use the VOB+BUP+IFO CREATED BY IFOEDIT (VTS_01_0 for Trailer Recommendations, VTS_03_0 for Trailer+MusicVideo, and VTS_06_0 for movie) to REPLACE the original IFOS/BUPS/VOBS.
10. I open IFOEDIT again, this time opening VIDEO_TS.IFO. I click GET VTS Sector to fix and correct all the changes.

11. Result:
a. Movie plays fine, menu plays fine, quality fine...
but...
b. trailers and music video NOT SYNC. Opening the VOB created by ifoedit also shows no sync.

After opening the original VTS_01_1.VOB and VTS_03_1.VOB with Vobrator, i found out the following:

VTS_01_1.VOB has 3 trailers, and between each there's a black mpeg2 still, so total of 7 PGCs (double still mpgs at end)
VTS_03_1.VOB has 2 trailers and 1 music video, with 2 black mpeg2 still in between, so total of 5 PGCs.

I remux the rejig file with rempeg2, and now the VOB is smooth and SYNC.

So i guess, there lies the problem with ifoedit muxing vob that originally contains mpeg2 still ?

The solution with rempeg2 muxing does not give me new IFO, so the next process of IFOEDIT - GET VTS Sectors is not correctly done.

Any ideas?

cheers
robshot

[TEB]

just a little question for a newb like me:
When u spesify eg: 70% in rejig does it reduce EVERY frame down to 70% flat? or doest it vary and the total is 70% ? is it possible to use this process to "shave" off the tops on a VBR peak?

best regards teb

[naoenxeosaco]

I noted previously any DVD titles not play fine with ifoedit muxing. For this cases, I proceed this way:
1. Execute DoItFast4U on DVd Title;
2. In ReAuthorist make some adjusts and click in "Author It!";
3. Rejid demultiplexed video VTSs and rename output M2V file;
4. Import script in scenarist;
5. IfoUpdate
6. Burn.

Like you says... "is smooth"....

Bye

[dragongodz]

mmgrover - just wondering if you tried the changes i suggested and if so then did you test it with 50% compression ?

[DMagic1]

Is there a way to get the m2v file into Maestro? If it was said before I can't find it. 15+ pages to look thru.

[robshot]

To update the issue i had with Ifoedit 0.97 does not mux rejigged m2v containing still mpeg2 pgc(s), the workaround is as follow:

1. with vobrator, identify which are the still mpeg2 pgc(s) in your original .VOB file.
2. using Ifoedit to open the corresponding .IFO, VOBIDsplit it to new sets of VOB(S).
3. rejig the VOB file(s).
4. when you have the vob file(s) with no still mpeg2 pgc(s), the new rejigged file (m2v(s)) will be muxed fine with Ifoedit
5. last stage is usual, open VIDEO_TS.IFO and get VTS sector.

I use MenuEdit to remove buttons and/or language menu(s) I don't want, and combining it with Ifoedit + Rejig makes a backup process without the need much of anything else.

To _again_ list the procedures:
1. SmartRip the DVD
2. VOBrator the VOBs to identify VOBIDs
3. Make decision to Rejig or not to Rejig which content material (should i delete the special features? Should I rejig some of them? What % calculation for movie rejigging?).
4. Ifoedit what VOBIDs and Stream needs to be stripped.
5. Use MenuEdit to remove buttons connecting to (removed) image menu and all
6. Ifoedit back the rejigged and MenuEdit-ed files. Check final DVD-R filesize.
7. Test playback of your VIDEO_TS folder with PowerDVD

Now, when would that requant engine get "smarter"? Transcoding now has a new name, but re-encoding with CCE and/or Procoder still give the best quality? It needs to be paralleled (sp?), right?

Rejig has given a high leap from the one-clicks transcoding quality, and it is nearing those of CCE and Procoder (engine 1.5).

cheers, nic and all!

[Nic]

Thanks for all the feedback robshot and everyone, I haven't disappeared just haven't been home much recently. Ill do a bug fix release ASAP.

@DMagic1: Your right, I will try and compile the useful things from this thread into text/doc. Unless anyone else would like to do it ? :)

-Nic

[mmgrover]

@dragongodz I tried what you recomended, didn't help much :-(

Different video streams on compress so small and
stop there :-(
I'm Open To Ideas??


@nic I hve been working with eyes` only and we have a version that
works well with DoItFast4U. Just having problems with getting
video to compress to a accurate size :-(

Let me put lates code on site for you to get.

It's At http://www.rejig.org/downloads/0.4b/rejig_0.4b.rar


It parses a REJIGData.txt file from DoItFast4U

mike


@robshot

Do You mind if I put your tips of wisdom on the ReJig web site?

[E-Male]

Originally posted by mmgrover
..we have a version that works well with DoItFast4U.....
music to my ears

[DMagic1]

Originally posted by Nic
Thanks for all the feedback robshot and everyone, I haven't disappeared just haven't been home much recently. Ill do a bug fix release ASAP.

@DMagic1: Your right, I will try and compile the useful things from this thread into text/doc. Unless anyone else would like to do it ? :)

-Nic

Until then could you or anyone else point me to the page its on if you know.

[Eyes`Only]

mmgrover: the last alpha I gave you works well then? I'll rls a DoItFast4U! 1.3.4 tommorrow (or soon) with the latest additions.

all: The integration between the two apps works well, we are still experiencing a problem with oversizing since it seems Rejig can only compress so far before it just doesn't compress anymore(?) currently. I don't see anything more I can add to version 1.3.4 to rectify this, and I think the solution will come from the Rejig development at this point.

[dragongodz]

mmgrover - ok i would attach one with a more extreme changed values but cant so message me where i can send it to you so you can give it a try. i quickly tested it on 2 files and 1 it came close but the other just would not go below a certain size. so you probably want to try it with a couple of different files. the changed source file is included and i have commented with dragongodz above the changes so they are easy to find.

[mmgrover]

@dragongodz

You can e-mail it to mike@rejig.org

mike

[E-Male]

can't wait to try full backups with the new versions of DIF4U and ReJig

[motster]

FWIW, the crashing seemed to go away when I went from daemon-tools 3.32 to 3.41. More info as I find out!

[echooff]

mmgrover


I know you guys are working hard on the code. You posted version 0.4b on your site. I am not a programmer and don't own a compiler. I also realize it's probably not ready for issue yet but...Could you possibly compile this version.

[robshot]

mmgrover wrote:
@robshot
Do You mind if I put your tips of wisdom on the ReJig web site?

well of course i don't mind, mmgrover! Surely this is an exciting thread and after out from the scene for quite a while, i got in just in time to read this thread. All i can say is WOW! Got me hooked rightaway :)
Perhaps later a more detailed but simpler guide would follow. So far, i got 2 perfect backups with (Smartripper + VOBrator) and rejig + ifoedit + EditMenu.

cheers mate!

[mmgrover]

@echooff

I uploaded a compiled exe
http://www.rejig.org/downloads/0.4b/ReJig.rar


@robshot

thanx, I'll get on that tonight :-)


mikw

[E-Male]

is this new version any better than the one before until the new DIF4U version is publicly released?

EDIT: found the new version

[E-Male]

ok, i ran DIF4U
then processed the result with ReJig
but how to go on?
(i tried reauthorist but it crashes)
can someone help me out?

thx
e-male

[dragongodz]

mmgrover - just had a quick try of 0.4b. it appears to use the changes for output size that i sent. so you got ok results with what i sen then i take it ?

[E-Male]

i didn't check the size
when the crashes happend and couldn't get further i just deleted the files
sorry

[Superduck]

Just tried DIF4U latest version with Rejig 0.4b. It seems that DIF4U set the compression ratio to 1% :/ So I had to do it manually afterwards to get it right.

[cweb]

I've taken the liberty to port this to Win32 so it can be compiled with MSVC++ and does not require the ugly Cygwin environment. Will run on Win9x/ME/NT/2K/XP/2003. I really don't deserve much praise for this as the port was fairly easy :)


Why not try the much nicer mingw environment? It's better than the ugly Cygwin.

[Nic]

@cweb: I assume you've taken that from the first post from RB. Alot has changed since then... ;) But leads me to another topic.
I've been playing with ffmpeg MPEG-2 Encoding in my absense, might try and include that into ReJig as well as normal Domain Transcoding...Has anyone had any good success with ffmpeg mpeg-2 encoding for dvd authoring?

Ill post my results of it as well as a cygwin build (it just doesn't build nicely in mingw and it still needed the mingw DLL mgwz.dll). (My build can take multiple .m2v inputs and output one single .m2v, which the original can't)

To bring it back on topic, I've fixed a few bugs in ReJig, will release that ASAP, then begin the rate control stuff.

-Nic

[winman]

@nic

Not sure about the ffmpeg command line version but I have great success with the ffmpeg MPEG-2 encoder that comes with ffvfw(Sep 27 2003 build). I mostly use it for capturing and transcoding divx materials. Quality is quite good and the speed is great (on a 2.0GHz Athlon system - a 640x480 video source get ~60+ fps w/o resize and ~45+ fps w/ resizing ).

The resulting m2v work fine with Maestro amd bbmpeg... Playback is also great as long as the resolution is compliance. Reducing the maximum I-frame interval to 18 or less would probably help with compliance but most hardware players I tested don't seem to care if you have more than that.

[DMagic1]

i'll wait for the answer

*question edit out*

[JvD]

Nic, "then begin the rate control stuff", OK, I think I heard this a LOONG time ago ;-) I have done several backups with Rejig now but mainly together with Big3-1 :-) It would be great if Rejig had it built in. I use ReAuthorist and the values thhat I get from there (including an control maually) witch end up around 4250 +/-100.

[Nic]

(what can I say? Im a busy and easily distracted guy ;) )

[DDogg]

Nic, great work. Since you brought up mpeg2,[I assume for situations where transcoding would not be appropriate?] might there be a possibility to also use the CCE plugin as an output module? I have tried a few tests with the experimental internal mpeg2 [of ffvfw] and show problems with blocking and strange spots on graphic slides. Perhaps just my method using VDub. If you will let me know what you want tested with the internal mpeg2 [of ffvfw], and how you want the results correlated, perhaps I can contribute to the effort. Here or PM is fine.

[DMagic1]

I asked a while back, how to get the Rejig M2V into Maestro. I never got and answer. What I did to get Maestro to accept it was to run Pulldown setting only "all frames to progressive". After that Maestro accepted it fine.

Is this the best way to get Maestro to accept the M2V file?

[dragongodz]

DDog - the mpeg encoding (especially mjpeg) in ffvfw is very buggy so dont take that as being typical. Nic did a little program some time ago (avs2mpg i think it was) that used mjpeg compression and it appear to work fine.

the only problem with mpeg encoding from ffmpeg is its GOP usage. that is if you just encode normally and even set the GOP size to 15 or 18 it doesnt strictly stick to it. example i have a clip that has a lot of movement in it. ffmpegs method to keep it looking good ? short GOPs. when the motion gets heavy it became IBBIBBIBBIBB etc. while this keeps a perceptial quality it is not that good for size and may stutter when you play these sections(this clip does stutter a bit for me).

a more reliable method would be to base the GOP off what the original is and recode accordingly. that way asuming the original encoding knew what it was doing any change in quality should be consistant. the exception being if the payback system i mentioned waaaay back was implamented. then high motion scenes would get extra bitrate anyway.

Nic - you asked about PSNR somewhere back there, ffmpeg has PSNR reporting for encoded frames if you are still interested.

mmgrover - you still around mike ? just wondering why you didnt release the source to 0.40b using the changes i made for output size. others could probably play with it more.

[Nic]

Thanks for that input dragongodz...Hadn't noticed the GOP problem. It's because of the scene change detection code in ffmpeg. If I remove it...things go back to being reasonably strict GOP sizes.

Keep the info coming :)

-Nic

ps
Actually I can raise the detection level so it occurs alot less....that is probably better than turning it off altogether...

pps
@DMagic1: Sorry if it seems im ignoring your question, but Ive never used Maestro. So have no idea, sorry, hope you get an answer to your question.

[echooff]

@mmgrover

Sorry for the late reply. Been under the waether. Many, many thanks for the compiled version. I love this tool.