GPL command line M2V transcoder

[dragongodz]

ok a quickish way to get some psnr results

create .d2v's with dvd2avi for original and rejiged .m2v's

create avisynth script like this

LoadPlugin("MPEG2Dec3.dll")
source=MPEG2Source("g:\test\original.d2v")
test=MPEG2Source("g:\test\rejiged.d2v")
test=converttoYUY2(test)
source=converttoYUY2(source)
compare(source,test,"","result.txt")

then load the script(.avs) in mediaplayerclassic(as example) and play.

you need nic's mpeg2dec3.dll aswell of course available from his website.

note i converted to yuy2 but you could convert to rgb aswell though you get lower values then. :)

not wholly satisfactory but a start. quick test with new engine versions showed slightly higher overall PSNR with wider range for my changes. quickly did old engine compared to new(both my latest changes versions) and old engine had slightly higher values.

really needs someone to do some full comparisons which i am afraid i dont have time for at the moment. i only tested a 3 second clip so not really anywhere near enough.

also there is the SSIM avisynth plugin which would be interesting to see what results that gave aswell(requires YV12).

[JvD]

Originally posted by dragongodz

really needs someone to do some full comparisons which i am afraid i dont have time for at the moment. i only tested a 3 second clip so not really anywhere near enough.


OK, tried your method Dragongodz and it seems to work better. I compared original m2v to a 1-pass rejiged file and Original to a 2-pass rejiged file. Its a 20 min part of a french movie Jean de Florette (1989 and not so good picture quality). The Rejifd files is at 80% compression. Its a queit slow move, not any wild action scenes. Below I cut and past the results.

-----------------------------------------------------------
Original vs 1-pass Rejig
Minimum Average Maximum
Mean Absolute Deviation: 0.0000 0.2388 1.6864
Mean Deviation: -0.2518 -0.0138 +0.0779
PSNR: 35.1470 70.6946 107.3187
Overall PSNR: 49.1752
-----------------------------------------------------------
Original vs 2-pass Rejig
Minimum Average Maximum
Mean Absolute Deviation: 0.0000 0.2108 2.3003
Mean Deviation: -0.2525 -0.0126 +0.2056
PSNR: 34.0277 71.8277 107.3187
Overall PSNR: 49.7356
-----------------------------------------------------------

I am not shure how you can make the test more efficient. If I run throug a full movie all extreme values get lost. Is it better to pick some hard parts of the movie and then do the comparisment?

[jdobbs]

I've always found it best to save the values and then import it and create a line chart in Excel. That way you can see the deviations graphically over time. The averages sometimes throw you off -- you can get huge variations in high-movement scenes and virtually none in low motion.

[E-Male]

does anyone know a source that gives more details on psnr?
because comparing a divx to itself give >105db and with undot it gives ~60 which is a huge difference for such i mild filter IMO

[dragongodz]

jdobbs - the script i gave above saves to the result.txt file ibcluding PSNR for every frame but even so the end pieces that JvD posted do show minimum value it got to etc.

what would be interesting is to take 3 30 second clips, A being all high motion, B being nearly static/low motion and C a combination high and low. then run each through comparing old and new engines etc plus comments about what they actually look like to the human eye.

E-Male try doing the same test using SSIM(its in the avisynth section somewhere) and see what that gives you. as i said SSIM is meant to be a better test anyway so if i get the time i will see if i can get that up and working for the .m2v's(unless some avisynth guru would like to ? :) ).

[JvD]

dragongodz, ill do that kind of testing later today. Have to spend some time with my family :-)

[E-Male]

tried the ssim plugin
values make much more sence

since the result is written to a log-file it would be great to have possibility to play the avs file at max speed without showign the picture
is that possible?

[JvD]

OK guys, i did som testing. Didnt have that much time but anyway. I put it all on a html-page for you to read (the adress i temporary so Ill chaning it later). I only had time (for now) for two scenes and with one and two pass on dragongodz latest modification. Also included som pics.
Here we go! (http://www.morck.se/psnr/)

[TheXung]

Does anyone know of any projects working on frequency domain filtering?

[dragongodz]

JvD - thats cool. i too have a family and generally only get near my pc at night when they have gone to bed or the occasional(like now) hour of an afternoon. interesting results though, not bad. how did the actual clips look to your own eys though ? any flickering, any noticable blocking ?

e-male - open the .d2v and there is a "framerate =" , just double the value after that and your clips should play double time. :)


hmm tried the ssim plugin and just get an access violation. what the ????

[JvD]

Hard to tell any difference between the two clips. Maybe the really rapid scenes looked better with the latest ReJig. In more calm and slow secens its really hard to tell the difference between the two, picturewise and I didnt experience any strange behaviour, no flickering. Will do some more testintg later (my only contributin since i dont have the skills in programming like you guys) on this week when I get more time.


PS. Why is it so quiet around here? Any new versions in sight? DS.

[JvD]

OK, didnt notice this before But when I try to use the ifomode and pick wanted streams (video, audio and subpic) and dont get subtitles to work the last part of the movie. Tried in two movies and whats them through on my computer, everything looks nice but somewhere in the second half the subs disappear. WHY? I author it back usingin ifoedit (+exported chaptimes). Havent noticed it before? Am I doing something wrong or is Rejig pulling my leg?

[dragongodz]

Jvd - ye new engine looks better to me for action scenes, smoother/less blocking. the last changes i made introduced a little blocking for increased P frame quality so may knock that down a touch and smooth it back out a bit.

quiet ? well Nic said he has been working on dvdauthor and muxxing subs etc so that would make him very busy. when he is happy with everything i am asuming rejig will jump to 0.5 as he hinted. :)

i will try out subs in the next few days and see if that happens for me aswell.

[jj59]

On page 28 of the thread Nic mentioned a bug in sup creation, probabely something with the change of a vob_id, he also mentioned he fixed the bug. So we have to wait for the fix in the next release.

[E-Male]

if yo ugot problems with authoring the dvd (including subs) in ifoedit try using "remux m2v" instead of "author new dvd"

[Nic]

@JVD: Yup theres was a problem in SUP Creation...when a PTS gets reset back, it throws things around. That should be accounted for in 0.5 (well it works in Matrix: Reloaded :) )
(Funny thing is, VobEdit doesn't extract the SUP right either...has the same problem, once I realised that, I didnt feel too bad ;) )

0.5 is nearly ready...far from perfect, but a nice start in the right direction. And I know where I want to take 0.6 now as well.
Just got to do more testing in 0.5 (as well as write a HTML helpme) and then ill release it (which means ill probably have this thread closed and start a fresh :) )

Cheers,
-Nic

[E-Male]

you could release the "almost ready" versiosn as alphas
if you tell what needs most testing we could try it, so you'd get results from variouse systems and discs
would be chance for people like me to contribute

[JvD]

Nic: Looking forward trying it out end testing it Nic. VERY excited to see it. Any adjustment in the engine or just those added features?
Nic and you other guys, yeas ive read about the problem now. Forgot about it, this tread IS long :D

[2COOL]

@E-Male

Originally posted by E-Male
if yo ugot problems with authoring the dvd (including subs) in ifoedit try using "remux m2v" instead of "author new dvd" From experience, I had audio delay issues remuxing with IfoEdit.

The only good part on using IfoEdit's remuxing is that you can remux the whole new m2v into the whole titleset's VOBs. But I get audio delays after playback.

For you to remux the m2v, the whole movie, with VobEdit, you would have to join the whole Titleset VOBs as one and then remux. Any stream stripping would have to be done prior to remuxing. It's a longer process when you have more than one titleset VOB. Of course, you could get the m2v from each VOB separately and then mux it back with VobEdit. If I had 5 VOBs to contend with, it can be really bothersome.

[E-Male]

well, i had no problems until now, but that's all i can say
everything i recommand is based on personal experience and might of course be incomplete or even incorrect

[2COOL]

Maybe you were working with movie with 0 audio delays. Have you tried Matrix Reloaded? That had audio delays of 100.

[E-Male]

of course it's possible that i was just lucky

don't have matrix reloaded, so can't try that

[robshot]

Here's another shot on trying to find what's wrong with ifoedit remux (if there is), and other stuff.

I've been collecting Stargate SG-1 R2 DVDs. Now I'm trying to do backups. SG-1 R2 discs are PAL, 4 eps per disc, with trailers, 4 audio tracks including director commentary for each episodes and all.
Basically 1 disc would be around 7 - 8 GB. I love MGM for not skimping :).
Doing a rejig for the episodes (in VTS_01_XXX.VOB), quality v.s. filesize, would need around 85% rejig. But still it won't fit 1 DVD-R since the motion menu (uglies as they are) would take around 300mb to 400mb.

Initially i was planning on backing up 1 dual layer to 2x DVD-R. But I explored the possibilities of putting all the eps, take out the trailers and documentaries. Still there's about 500mbs overheads. So, i decided to REPLACE the motion menu with STILL MENU.

Ok, now the process:

Ifoedit remux "hates" it when there's a still VOBIDs in your movie VTSs. In SG-1 movie VTSs, the first VTS_01_1.VOB has 4 STILL VOBIDs. This need to be taken out, and i use IFOEDIT VOBID stripping. All goes well afterwards. Rejigging the 8 VTSs, then remuxing with IFOEdit, creates sync a/v. (The same thing I did with Region 1 RIDE WITH THE DEVIL - where i kind of FOUND OUT the ifoedit muxing problem lies from this STILL VOBIDs).

All set with the rejigged episodes (yeah, quality nice nice), the 300+mb overhead challenged me. They're the motion menus! This, requires MenuEDIT, VOBRATOR, VOBEDIT and REMPEG2 :).

I open the VTS_01_0.VOB (the menus), and voila, 32 VOBIDs, with some has multiple CELLS.
I then open a copy of the same file with VOBRATOR, for analysis. I found the VOBIDs i wanted to take out. I extracted the VOBIDs with VOBRATOR, and REPLACE the video stream with STILL VIDEO i made using REMPEG2 (under the mpeg2 tools). This new STILL VOBID needs to be stripped of its audio, so i open VOBRATOR again, then extract that VOB without the audio. Voila, i got a perfect NEW STILL VOBID complete with Subpictures.

This NEW VOBIDs then got injected with MenuEDIT. Now, there's also issue when you want to inject CELLID. Same as above, extract the VOBID with VOBRATOR, then open the VOBID (which contains at least 2 CELLIDs) with VOBEDIT. Demux by CELLID, and you get VOB files of the CELLIDs. Insert this into MenuEdit project, and fix whatever button you need to fix with menuedit.

While this seems complicated, I think i've solved the ifoedit remuxing issue, and at the same time, i was able to find a way to replace motion menu with still menu without so much as reauthoring. the only time i need to encode is when i want to make the new STILL MENU to replace the motion menu (filesize consideration).

I don't have matrix reloaded with me to test, but some of the nasty anime DVDs i have, and the Region 1 (NTSC) discs (TNG BOX SETS), are now muxed correctly with ifoedit.

So, in short:
1 Dual-Layer PAL/NTSC with Motion Menus ---> 1 DVD-R PAL/NTSC STILL MENU:

1. ReJig (hehehe... part I)
2. IFOEdit (hehehe... part II)
3. MenuEdit (menu manipulations and pic grabber for still menu)
4. Vobrator (analysis and vobid extraction)
5. VOBEdit (demuxing by CELLID)
6. ReMpeg2 (remux still m2v to existing VOBID)
7. Any mpeg2 encoder for making still menu.

I'm thinking of making how-to, but it's already morning.

cheers
robshot

[robshot]

2COOL(the nick says it all),
Got a question that i haven't been able to answer after reading your many-wonderful Cheatlists :)

You ever encounter R2 discs where the first menu is Language options for multiple language MENUS? For example, Brotherhood of the Wolves (2 language options French and English - will reroute to specific DVD MENU in French or in English). or the MANY Stargate SG-1 Region 2 (from UK) DVDs.
Is there a way to skip this (and use english as default language menu) using IFOEdit? So far my tests failed (based on the cheatlists), so i still have to put this on the backups (but i did replace the motion menu from and after this language option, to still menu. bitbudget for backups are important factor).

cheers
robshot

[VILLA21]

@VILLA21: Anyone tried the latest DoItFast4U with Rejig? I,always, get oversized DVD files in scenarist, around 4800Mb.

@Eyes`Only: Yeah, I don't know when they're going to have that fixed. DoItFast4U! passes the correct values to Rejig, but the output from Rejig doesn't match the size that it should with the parameters that are passed if the bitrate is low (ie. extras). Hopefully someone fixes it soon, there's nothing I can do about it.

So, is there any solution so far?

[E-Male]

i did some size tests and what you can take for sure is this:
no matter how compressable the source is (i took Ice Age as ideal source) going below 30% in rejig won't work, stuff hardly get's that small, so if DIF4U try to compress some high bitrate extras to very low bitrates i might cause the trauble with this
just an idea

[mmgrover]

Hi all...

I'm still here, Just In Boston right now :-)

I havn't forgot about this!

mike

[dragongodz]

mike - nice to hear from you. :)


what you must understand for high amount of compression there is a saturation point. that is a point where all quants for B frames and all coefficents etc are already being done. once that point is reached rejig can not currently do any more. yes there are ways to reduce even more such as skipped frames etc but the current engines do not support that. they are really designed for lighter compression.

[Fabiano]

Hi, is there a reason for Rejig not compress HDTV MPEG-2 ? :confused:

Rejig just removes stuffing but does not compress at all.

I tried 1920x1080i and 1280x720p, but no luck!

[Nic]

An email from Makira himself (creator of the transcoding engine):

I've just read the doom9 thread. Unfortunatly, I can't reply ("You are not allowed
to post
or reply until you have been registered for at least 5 days.").

I've seen the screenshot sent by JvD. The point is, the new engine will make some
frames
look worse than the old. But it will also make bad looking frames better. In other
words,
the goal is not to have a big gain in overall psnr, but to reduce the variance of
the psnr.

So we take bits from some pictures and give them to others. This is most noticeable
with
a high motion scene followed by a very slow motion scene. With the old engine the
first
scene will look bad, but the second will be perfect. With the new engine, the first
will be
much better, at the detriment of the next. The overall quality experience should be
better.

Here's a very interesting read: "Two-pass mpeg2 vbr encoding" research paper by ibm
http://www.research.ibm.com/journal/rd/434/westerink.html

Let me quote an important part:

" Apparently, the overall visual quality of a video is rated not so much by the high
quality
of certain scenes, but more by the scenes of the lowest visual quality, the ones that
might show obvious artifacts and thus stand out."

This is the main basis for the changes between the old and new engine. That being
said,
there is a lot of room for experimentation. If I had more spare time, I would love
to clean
m2vrequantiser's code and make it easier to test different algorithms. With my current
schedule, that'll have to wait till after christmas.

Feel free to repost this on the thread, since I can't.

Regards,
Makira

[dragongodz]

yep which just basically backs up all i said before, old engine is better quality static or slow/low motion scenes while new engine handles high motion better. :)

problem was that too much degradation was put on P frames and showed up blocks from them rather than the B frames (which is what the old engine suffers from). thats what my experimentation with both engines has been all about. check the old engine in my mod 3 and it looks better(less blocking) than the original and the new engine doesnt suffer from P frame blocks but loses some sizing accuracy when compressing below 70%.

i will make a couple of more slight modifications(such as upping the P frame compression level a bit) for that bit extra compression space etc. though you may have to wait for rejig 0.5 to try it out. :devil:

[Amnon82]

Hi Guys!

Thanks to all who created ReJig!

I want to do some to get ReJig in Germany also popular like here.:sly:

So I translated it to german.

You can find the modded version on my Homepage (http://www.dvdrpage.dl.am)

I used Nic's 0.4i version.

Also I found a bug. Maybe it is not ReJig's fault. (But I think so!)

I always got 'Too many frame drops' in IFOEdit 0.96.
So I started to fix it.

Here is the way to solve this problem:

Rip the DVD in IFOmode in ReJig without the subs.

Open VOBSub Configure and klick on 'Open'.
Open the IFO of the mainmovie and klick twice 'OK'.

VobSub is no indexing all subs.

Open SUB2SUP and choose the idx-file created by VOBSUB.
SUB2SUP convert now all subs to sups.

Author now Your DVD with IFOEdit with the new subs.
Now it will work.

I hope this will be fixed soon...

[Amnon82]

Here is the BugFix 0.99a for ReJig v.0.4i-de:sly:

Download file (http://rj-elektronik.de/avideo/files/BUGFIX099a.exe)

[Amnon82]

Now is the german Guide for ReJig v.0.4i-de final!

:p Avalon's german ReJig Guide (http://rj-elektronik.de/avideo/main.php?set=shotan)

[Nic]

Ive created a new thread for ReJig, so this one can be closed. It's too mammoth for beginners to get into and should probably be closed. Please post new posts into the ReJig 0.5 thread. Thanks :)

@Amnon82: Thanks for your updates & page, haven't been able to look into it properly yet for 0.5....

-Nic