Mencoder. Trying to enocde DVD-Compliant Mpeg-2.

[video_magic]

Hello, thanks in advance for any help anyone can give me.

I have got:

Windows XP Pro SP1 and all Windows updates (except SP2)
Pentium 3 1.1 gig
half gig of ram
120 GB hard drive space using NTFS

I had a wedding video on camcorder which I wanted to capture losslessly onto my computer and then put it onto a DVD-r

I read around a lot, then I captured it using HuffYuv at 768x576 which I thought was 'full pal' to capture all the detail, so I now have it as a 35.3 GB AVI file. It lasts 1 hour 5 minutes. It has audio too but I am happy to demux that and upsample it to 48000 hz using SSRC, Then remux.

It's going to be quite important for the person who will have this DVD to want to pause it at any Frame, and preserving as closely as reasonably possible the picture quality of the original tape.

I had been reading up and trying Mencoder for weeks! I have got it so the quality of the picture (pausing at different frames too)is all I could want, but the program just will not respect the maximum bitrate! I would be quite happy to do a CBR encode but that also does not seem possible as far as I have experienced. I have read through the man_page.html over and over, I have tried so many different combinations of switches!

I do not want to use B-Frames, and I want the bitrate to be around 7200, I do not use Noise Reduction nor any picture quality filters. I resize to 720x576, I also have a problem with making sure the aspect is seen as '4:3' on the encoded files. Media Player Classic always plays the files though the aspect looks too square at 100% zoom, and too widescreen if I double the size, though the picture looks fine. Muxing or Multi-plexing or whatever it's called in different programs warns me of the exceeded bitrate, or I get an incorrect aspect warning.

Here is a latest script that also doesn't work (bitrate spikes)
C:\asblah\mplayer100105P3CD\mencoder.exe -vc huffyuv -aspect 1.3333 -nosound -vf format=yuy2,scale=720:576:0:0:0.00:0.50,dsize=4/3,harddup -sws 2 -noskip -of rawvideo -ofps 25 -ovc lavc -skiplimit 0 -lavcopts vcodec=mpeg2video:vme=0:mbd=2:keyint=0:sc_threshold=1000000:vrc_eq=1:vrc_init_cplx=1000:vbitrate=7200:vratetol=1100:vrc_maxrate=8000:vrc_buf_size=1835:vfdct=2:idct=3:nr=0_mask=1.0:dc=11 d:\capvid\matwed1min53.avi -o d:\capvid\mtwd

I have tried 3 different recent compiles of mplayer bundle, including Celtic Druids' , I have tried different rate control methods 'vrc_eq=' , I have recent FFDshow compile installed and it doesn't seem to make a difference if I use that for decoding Huffyuv rather than force proper HuffYuv with '-vc huffyuv'. I have tried & tried so many different things, but is it possible to produce a DVD compliant Mpeg2 encode at around 7200? Is it possible to force a CBR encode?, I suppose I don't mind whether I do a 1 pass or 2 pass.

Is there also a program somewhere that can tell the colour of a file (I mean like whether it's YV12, YUY2 .....)?

An error I always get while encoding is lots of rc buffer underflowmin scrolling down the Dos window

THANKYOU for any help, I've got a real headache here. Quenc encodes including todays 0.59b also exceed bitrate. AviDemux latest 2036 gives me invalid files on DVD mpeg 2 encodes. I'm really stuck on this!

[evade]

I have tried to produce DVD-Compliant streams with mencoder under linux and run into the same problems.

I would reccomend you try Peter Cheat's modified mencoder located at
http://petercheat.host.sk/libav/ . It should respect minimum/maximum bitrate.

[mean]

For avidemux2, try settings the output to mpegV and use DVD as codec
That way you will have video only
Ditto for audio, mux using mplex and you wil have a 100% DVD compliant
at the end.

The only know drawback is DVD codec (based on mpeg2enc) is slow.

[video_magic]

Mean:
, I just get the same thing when I try encoding to DVD or DVD (lavc)

It gives me files that are huge (like almost as big as the Huffyuv input clip), and that cannot be rendered or muxed. Gspot cannot identify them either.

This is using the latest version. 2036

Evade: I am slowly getting the Peter Cheat Mencoder and will give it a try.

Thanks guys both for suggestions. I think there is definitly a rate control issue that is the only thing spoiling this Mpeg2 encoding , because the results themselves look very good, I just cant get something I can put on DVD (yet!)

I will not give up on this tho'

[mean]

Did you press the vprocess button ?

[video_magic]

OMG! That worked Mean, thankyou thankyou thankyou!!!
I had pressed it before but no menu popped up and so I had just thought I shouldn't have pressed it

Now I know!

The resulting m2v is great, it plays nicely and I am able to mux it (with Peter Cheats mplex 2) with NO errors, the bit-rate is within the limits I set and Gspot identifies it and with the correct DAR 4:3 !! Everything is fine now thanks to you Mean after I had tried so many things. You are wonderful, thanks again

[video_magic]

Originally posted by video_magic
........ Is there also a program somewhere that can tell the colour of a file (I mean like whether it's YV12, YUY2 .....)?
...

I figured this out now. I can use avisynth, make a little AVS script file like this:

OpenDMLSource("D:\capvid\mtwd1mnnoblue.avi", fourCC="HFYU")
info()

When I play this AVS in a program like Zoomplayer or Media Player Classic it displays information including the 'colourspace' of the file on the screen!

[video_magic]

I had got Peter Cheats' modified mencoder and tried it using an 80 second test clip of my Huffyuv avi.

I tried lots of different settings and got it so that it would give me both a DVD compliant m2v (mpeg2) and so that I was very pleased with the quality!
But when I set it up for the full clip (1 hour 5 minutes 35.3 GBs) it would only encode the first approx 100 seconds of it! It was incredibly frustrating to have got so far then to be stuck at another obstacle! I tried different things with the -endpos switch using the time and the mb options but same thing.

Then after thinking some more I realised it could be the openDML AVI of greater than 2GBs or 1GBs or whatever the limit is. So I then searched and got a version of mencoder with avs support from here
http://oss.netfarm.it/mplayer-win32.php as I can only think of avisynth of being a way to get round the limit (don't know how else to tell mencoder to accept an avi bigger than 2gbs?)

I am trying it now but it has a long time to go! I think it will probably have the rate control problems though as it's just built on cvs 17-01-2005 .......


I wonder if there is a version of mencoder that both has rate control fixes and will accept an avs?

[evade]

Peter Cheat's Nuenc is based on the same libavcodec with his new rate control and will accept avs. Different commandline than Mencoder though.

[Bogalvator]

To use avs files with Mencoder, I use akupenguin's AVS2YUV:
http://students.washington.edu/loren...synth/avs2yuv/

It's designed for Linux use with Wine, but it works fine under Windows. Use command line:
avs2yuv "z:\blah.avs" - | "z:\...\mencoder" - -o "z:\...\whatever.mpg" -of mpeg -ovc etc

So this will work with Peter Cheat's Mencoder too.

[video_magic]

Thankyou evade,
I have been trying nuenc from the commandline today. It gives nice quality results as always; however I have not got it to respect the maximum bitrate (get big spikes) although I have had nil buffer underflows which is something!!
I think this could be different rate-control somehow, because in the normal encoder of his it says about special options for pass 1 which aren't available AFAIK here (as in his important.txt). Also it seems to be letting me use MB >0 and still use the SATD etc options which is different from what mencoder documentation says. Also in Peters' normal mencoder (not the nuenc I mean)it has the pass numbering normal, whereas in the ordinary mencoder it says use pass 1 for first pass and pass 3 for passes 2 and 3, I tried both with Peters mencoder and both seem to work as far as I can tell.
However with nuenc from the commandline I still get exceeded bitrate anyway so I am keeping at it, the results as always do look great though

Also a link "The Truth About Maximum Bitrate" makes we wonder if I have a problem at all so long as no buffer underflows occurred even if I get Bitrate spikes? Can anyone explain or confirm, please see below, thanks
I cannot find the page online for "The Truth About Maximum Bitrate" but I had saved it so I paste it here (hope that's ok)

----------------
The Truth About Maximum Bitrate

This document was created to clear up misinterpretation of maximum bitrate in the context of MPEG-2 (VBR) encoding. The maximum bitrate is NOT the maximum instantaneous bitrate of an MPEG stream. It is not the maximum-average bitrate over a GOP or one second of the sequence. The maximum bitrate STRICTLY means the bitrate required in order to keep the VBV buffer from underflowing. A buffer underflow occurs when the decoder requests a frame, but there is no complete frame in the buffer. The MPEG specifications require a frame to be completely loaded into the buffer before it can be transferred to the decoder. The "maximum bitrate" is the maximum bitrate at which the data can be transferred to the VBV buffer. It is the bitrate (or datarate) coming from the disc to the buffer. The maximum bitrate possible to a MP@ML compliant VBV buffer is 15Mbps. Maximum bitrate does not correspond to the maximum bitrate of the reordered stream.

It might be easier to understand this using a tap (TAP1), a bucket with a hole and water, where the bucket represents the VBV buffer, the water represents the MPEG stream, and the tap controls the water flow to the bucket. Initially, the bucket (VBV buffer) is plugged up and filled with water until it is full. When it is full, the hole is unplugged and the water leaks out (the stream is going to the decoder) The tap is regulated so that it does not overflow the bucket (VBV buffer). Provided that the bucket always contains water, there will be no problem with the stream (CBR). Now imagine that the hole in the bucket is made larger than the hole in the tap, and a second, larger tap (TAP2) is placed over the hole. The stream to the decoder can now be regulated with the second tap (VBR). Clearly, the stream will continue to flow to the decoder as long as there is water in the bucket. TAP2 can be opened completely for sometime, without causing the bucket to completely empty. But if the bucket is completely emptied, the stream will stop flowing to the decoder. This is an underflow.

For those that do not believe that tis information is correct, or want to see the technical information, the MPEG-2 draft can be found here. The pages relating to VBV buffer (and maximum bitrate) are 170-175.

VBR allows you to improve the quality of an MPEG-2 stream. How much improvement depends on the VBV buffer size mostly, but obviously increasing the maximum bitrate will help a little as well (mostly in high action/motion sequences where the buffer is very quickly drained). In DVD and SVCD, the maximum bitrate corresponds to how fast the data can be read from the disc, its dependant on how fast the disc spins. Now some bright spark is going to point out that for a DVD player to be compliant it "must be able to sustain a bitrate of 9800kbits/s for an indefinite amount of time" (quoting mpeg.org). This is the maximum bitrate at which the video is read from a DVD. This is really only relevant for CBR encoding at 9800kbits/s. If a DVD player is not capable of sustaining this rate for CBR, the buffer will slowly empty out (consider a tap and bucket where the tap occasionally turns off momentarily), and eventually underflow. If a sequence was encoded at 10000kbits/s CBR, the DVD player will only play a very short part correctly (just over 4 seconds) before an underflow. Generally the result is jerky playback. In actual fact, a buffer underflow can also be interpreted as a maximum bitrate exceedtion, as the bitrate needs to be exceeded to avoid underflow (ie. the disc needs to physically rotate faster to keep up the flow of data).

There are other causes for jerky playback. I've personally found that streams created outside TMPEnc and muxed with the TMPEnc muxing tool are out of sync (easily corrected), or stutter during playback (can be a problem). Looking at such a muxed file in a hex editor, it is easy to see that there is a problem. Large sections of 0's are added as stuffing (padding) to achieve some minimum bitrate (300kbits/s I think?). This appears to be the problem (if you specify a minimum bitrate less than that used in TMPEnc muxing tool). bbMPEG (or MPLEX) is a better tool for muxing as it allows the user to make some selections. Remuxing streams that playback incorrectly, but reportedly did not cause underflow, solves the problem at least 90% of the time (it did for me).
-------------------------------------

Thanks, I am going to also give this a go Bogalvator (and thanks akupenguin). I am getting there I hope!

[hank315]

The Truth About Maximum Bitrate
This document was created to clear up misinterpretation of maximum bitrate.....

That's nice analogy and absolutely true!

Bitrate spikes are allowed as long as the VBV isn't "underflowed".
If your multiplexing program reports no underflows everything should be OK.

But even if a program like Bitrateviewer shows no spikes then you can underflow the VBV.
Bitrateviewer just calculates the average bitrate/second, this gives you a general idea what the bitrate looks like but if you have a few large frames in that second an underflow can occur.

You should try this:
Encode a short (20 sec) clip with a high bitrate (7000 kb/s) and load it in Bitrateviewer.
Open the video in a Hex editor and delete the first GOP, how to do that:
the file should begin with 00 00 01 B3, search for the next 00 00 01 B3 and delete everything before that string, now the first GOP is gone... save it and load it again in Bitrateviewer.
It will look different but it is the same video only the first GOP (12-15 frames) is deleted (+/- 0.5 sec. video).

Just want to make clear that it is the encoder that should make the bitstream DVD compliant by controlling the bittraffic in and out the VBV buffer.

Maybe you should take a look at my post tonight and try my (new) encoder

[video_magic]

Little update: I have just read that my DC value of 11 would not be compliant for DVD, values of 8, 9 or 10 are. I am learning stuff all the time!

I hope mencoder documentation will have some more lines of text added to it; it would just be a nice thing to have some hints and brief explanations for the user, regarding the settings and values.

Just a few extra lines and examples make the world of difference to the user when reading the usage documentation!

I will be getting back to trying mencoder with AVS input as in Bogalvators' post, but my PC is quite slow and I am giving mencoder a break and trying other mpeg2 encoders, I hope there might be an update from Peter Cheat with the rate control which would probably be the crucial thing to get an DVD compliant m2v out.

[video_magic]

Hey Bogalvator and akupenguin :)

I have now got the computer free and am trying Petercheats' modified rate-control mencoder (from November 2004), with using Avs2yuv to input from an AVS as mentioned above.

I am trying a test clip of 1minute 28seconds first, using 3 passes.

It took a very long time to get it setup from a batch file, and from the messages it seemed that it couldn't locate my files. Partly it looked like it was trying to use 'forward slashes' in the paths, whereas in windows there are backslashes. Anyway, I got it working by putting the avs2yuv.exe (version 0.23) and the avs file in the same location as the mencoder.exe and it's running pass 1 now. I am running everything from a batch file, my computer encodes quite slow at about 2 fps with my settings. The command between passes puts a pause on encoding of about 12 seconds. Anyway, this is all looking very promising THANKYOU A LOT for the help and the program! :)

title pass1 begins

avs2yuv "matp3t1.avs" - | "mencoder.exe" -nosound -noautoexpand -vf harddup -noskip - -o "d:\capvid\matp3t1.m2v" -of rawvideo -ofps 25 -ovc lavc -skiplimit 0 -lavcopts vcodec=mpeg2video:lmin=2.1:lmax=42.0:vqmin=2:vqmax=18:vmax_b_frames=0:vqdiff=3:vme=4:mbd=2:keyint=3:sc_threshold=-0340000:vbitrate=6800:vratetol=13000:vrc_maxrate=7800:vrc_minrate=4000:vrc_buf_size=1835:vpass=1:vi_qfactor=1.00:vi_qoffset=0.00:vqblur=0.0:vqcomp=0.5:vrc_init_cplx=1000:vfdct=6:idct=1:mv0:ildctcmp=2:precmp=2:cmp=2:subcmp=2:predia=6:dia=6:trell:cbp:preme=2:nr=0:dc=10:"

title 0

for /l %%a in (1,1,10400) do title %%a

title pass2 begins

avs2yuv "matp3t1.avs" - | "mencoder.exe" -nosound -noautoexpand -vf harddup -noskip - -o "d:\capvid\matp3t1.m2v" -of rawvideo -ofps 25 -ovc lavc -skiplimit 0 -lavcopts vcodec=mpeg2video:lmin=2.1:lmax=42.0:vqmin=2:vqmax=18:vmax_b_frames=0:vqdiff=3:vme=4:mbd=2:keyint=3:sc_threshold=-0340000:vbitrate=6800:vratetol=7000:vrc_maxrate=7800:vrc_minrate=4000:vrc_buf_size=1835:vpass=2:vi_qfactor=0.72:vi_qoffset=0.36:vqblur=0.0:vqcomp=0.5:vfdct=6:idct=1:mv0:ildctcmp=2:precmp=2:cmp=2:subcmp=2:predia=6:dia=6:trell:cbp:preme=2:nr=0:dc=10:"

title 0

for /l %%a in (1,1,10400) do title %%a

title pass 3 begins

avs2yuv "matp3t1.avs" - | "mencoder.exe" -nosound -noautoexpand -vf harddup -noskip - -o "d:\capvid\matp3t1.m2v" -of rawvideo -ofps 25 -ovc lavc -skiplimit 0 -lavcopts vcodec=mpeg2video:lmin=2.1:lmax=42.0:vqmin=2:vqmax=18:vmax_b_frames=0:vqdiff=3:vme=4:mbd=2:keyint=3:sc_threshold=-0340000:vbitrate=6800:vratetol=7000:vrc_maxrate=7800:vrc_minrate=4000:vrc_buf_size=1835:vpass=3:vi_qfactor=0.72:vi_qoffset=0.36:vqblur=0.0:vqcomp=0.5:vfdct=6:idct=1:mv0:ildctcmp=2:precmp=2:cmp=2:subcmp=2:predia=6:dia=6:trell:cbp:preme=2:nr=0:dc=10:"

[video_magic]

It worked great!

I then tried it with a higher vbitrate of 7000 and vrc_maxrate of 8000 and lower vrc_minrate of 3000.

That also worked just fine! NO bufferunderflows! Max Bitrate respected!, Picture looks spot on!

I think I was pretty lucky to have finally got the right combination of switches that work correctly with the rate control and also allows me to change the bitrate around without reverting back to underflows and spikes. Now the excellent AvstoYuv allows me to input via avisynth I'm very happy.

I don't know if Peter Cheats' rate-control and this support for avisynth could be worked in to Mencoder, it's very good and has really helped me. There are surely other people who would like it.

Now I'm setting up the full clip which will probably take a day and a half I estimate.

Since I have visited Doom9 wanting to encode to Mpeg2 for DVD I have ended up with three solutions to pick from- very nice, I'm very happy and appreciative


EDIT: I thought I should make it clearer, that Muxman (by MPUCoder)v0.11b accepts and processes the M2V and reports no underflows. This is how I tested to check if DVD compliant mpeg2