Is converting DVDs to AVC a good idea?

[comomolo]

I've tried a few times the search string "DVD to AVC" and I get no results. It's weird.

H.264 AVC is the best quality compression scheme available, right? However, most so called DVD-Backup solutions still rely on either stronger recompressing MPEG2 (DVD9 to DVD5 solutions) or transcoding MPEG2 to Divx/Xvid (DVD to CD solutions).

While I've found some solutions to transcode DVD to AVC (Nero Recode, for instance) I can't see much enthusiasm about it. Isn't it a good idea to transcode DVDs into the new H.264 AVC format? Shouldn't it be the less destructive of all transcoding solutions? Isn't it reasonable to expect a better quality than transcoding to ASP or Divx, for instance?

[82ross]

It is reasonable, but the time it takes to encode a dvd to avc is a lot longer than to encode to xvid for example. Look into it and you can see its just a matter of convenience, people rip to xvid a tried and tested method. AVC is great but is the time increase worth the quality increase?


For most people at this time, no. It is something that will progress though. Codec optimisations, better hardware to encode and better hardware to decode will all increase the popularity of avc.

[smok3]

well, iam very entuziastic about it, however i do prefer to keep DVD backups with all the 'makingof' stuff as well, nothing easier/faster than to fire some dvdshrink.

The mp4 authoring tools should really get better/accesible and it is a win then.

[comomolo]

I was asking because I want to build a media server at home and I was thinking about 1 GB per movie files. But I also like all the extra stuff that comes with DVDs as well as all the audios and subtitles. Isn't all that stuff available after transcoding?

I really don't care that much about the processing time. It's the computer working, not me... :) Is there any comparison of the quality I can expect from AVC vs Xvid?

[markrb]

I am doing much as you are thinking of now.
However my goal is basically 1/2 the orginal movie size, not the DVD size.
I remove all the extra's and then calculate what target size I go for based on that. A short movie might hit 1.4Gb's in size while a very long movie might be almost 3gb's. Large I know, but still a considerable savings over the 7-8gb's of the original DVD.
I put all the files on a server upstairs, connected by gigabit to the HTPC downstairs. Streams fine with no issues.
Some movies I do want the whole disc ready and those I keep in ISO format on the local HTPC drive.
I could always hit the closet and get the DVD out and put it into the player, but I love the idea of just a click away always.

I have tried Xvid, but I prefer X264. I use Megui as the frontend after ripping in IFO mode with Decrypter.

I have tried Nero Digital and was very dissapointed with the quality.

It all works really well with my setup. I use MediaPortal as the frontend of my HTPC and I use Media Player Classic to watch the MKV files.

Mark

[comomolo]

So you would say X264 is worth the processing time, right?

EDIT: I have found this: http://www.xile.net/xvid_vs_x264/

If that's what can be generally expected, I believe x264 is worth it.

[bond]

seeing that the avc forum has more posts already than the divx forum, altough its around for a far shorter period, tells me that there is indeed a lot of enthusiasm about avc :D

[markrb]

My only issue with x264 is the background blocks, but this also happens with Xvid.
It seems to amplify the source blockyness to my eyes.
I have tried just about every suggestion I have read and researched and nothing seems to help much.
If you are looking for true 1:1 quality I don't see it at this point with either of these, but to my eyes I prefer x264.
Although for smaller movies I am considering leaving them alone for now.

Mark

[comomolo]

Mark: since your goals and mine seem to be close, may I ask what MeGUI preset (if any) are you using to encode your pictures? Does this blockiness happen even if you use HQ-insane?

[Blue_MiSfit]

Keeping the extras on the DVD is tricky as this requires menus. Supposedly there is a program that can embed menu structure into an MP4, but I have never really heard anything about it.

I do 1.1 GB rips (1/4 DVDR) for most movies, and 2.2GB rips (1/2 DVDR) for longer movies, and those with exceptional quality that I want to preserve. This also affords keeping the AC3 track as opposed to transcoding to AAC.

1/4 DVDR with 6ch AAC usually results in excellent quality, indistinguishable from the DVD on an NTSC TV. The blocky background issue has been mostly resolved by a couple of things:

1) no fast p skip
2) disabling the in-loop filter (for 2.2 GB rips only)
3) using M4G's high detail matrix
4) careful pre-filtering to increase compressibilty (usually removegrain mode 5 or 2, or tweaked fft3dgpu + de-banding)

1/2 DVDR is exceptional. Transparent I would submit. This is between 1/2 and 1/4 the size of the original DVD, which is excellent compression IMO.

My usual settings off the top of my head:
*all motion search options
*deblocker -2,-1
*3 bframes, all options
*6 refs
*level 1 RDO
*level 1 trellis
*no fast p skip, no dct decimation

This is a tweaked version of the HQ-Slow profile. I usually get ~5fps on my Athlon64 (see sig for details)

XviD can pretty much do the same at these bitrates and is easier to decode, but I'm all about x264 these days. MeGUI is my best friend!

BTW, I usually add a tiny ammount of noise on playback with ffdshow to increase percieved detail. Luma<12, Chroma<8... Does a great job!


~MiSfit

[frodeste]

@Blue_MiSfit
:goodpost:

Could you tell me what kind of container you use that allows you not to transcode the ac3 or dts tracks? As far as I know, the .mp4 container does not support this.

One other thing I am testing, is the "threads option" in meGUI's profile for video encoding. By default, it is set to 1. I have a dual cpu setup, so I thought I would try to set it to 2. So far, this seems to speed things up. Comments?

[comomolo]

Would you say x264 at 1/4 the original DVD size is better than recoding MPEG2 to 1/2 the original size?

[bond]

Would you say x264 at 1/4 the original DVD size is better than recoding MPEG2 to 1/2 the original size?no, such a statement cant be done

apart from that mpeg-2 is not a codec, its a standard

[shon3i]

no, such a statement cant be doneBut any mpeg4 codec both asp and avc are better than mpeg2 at same bitrate almost 40-60%, this percent are grow up in small bitrates and lower in higer bitrates, like aac vs mp3

[comomolo]

Bond: let me rephrase:

If highest quality was your main concern, would you rather recode a DVD9 into DVD5 or would you recode it using x264 into something like half a DVD5?

And while we're at it, how much of a difference would do AVC vs ASP in this particular case?

[bond]

But any mpeg4 codec both asp and avc are better than mpeg2 at same bitrate almost 40-60%, this percent are grow up in small bitrates and lower in higer bitrates, like aac vs mp3not true, there can be easily avc or aac codecs that are worse then mp3 or asp or mpeg2 codecs

If highest quality was your main concern, would you rather recode a DVD9 into DVD5 or would you recode it using x264 into something like half a DVD5?i dont know. it not only depends on the mpeg-2 codec you use, but also on the video stream compressibility. i would say you cant answer this unless you try it yourself

if you say you would want to encode with a good mpeg-2 codec or x264 at the same bitrate i would choose x264.

And while we're at it, how much of a difference would do AVC vs ASP in this particular case?the higher the bitrate the less the difference. also it depends on the quality of the codec. talking only about the standards is misleading. a great asp codec, like xvid, could beat a crappy avc codec easily

it also depends on where you want to play the stream. if you want to play it on the dvd player in your living room you might want to encode to dvd even if the quality might be slightly worse.
if you have a crappy pc you might want to encode to asp because you cant play avc in realtime
aso aso


all in all we have rule 12 not for nothing. there simply is no "best" solution

[comomolo]

the higher the bitrate the less the difference
Would you give me a hint on where, more or less, would lie the frontier between the two codecs? 1000? 3000? 6000? (kbps) (I'm always talking about x264 and xvid).

all in all we have rule 12 not for nothing. there simply is no "best" solution
This is a beginner asking for experts opinion. If that's not allowed in this forum, please let me know and I will stop asking. I find your replies so far very valuable and helpful, as they're helping me understanding which way should I go, but if I'm in the wrong place asking wrong questions I'll understand and leave.

[markrb]

Mark: since your goals and mine seem to be close, may I ask what MeGUI preset (if any) are you using to encode your pictures? Does this blockiness happen even if you use HQ-insane?

This blockyness happens no matter the profile or even the bitrate used.
I am always trying new settings to see their effect and haven't come up with a solution yet that I am happy with.
I am leaning now to a smoother image. If I am right this should reduce the background blocks, but will slightly blur the image. I prefer a very sharp image, but I believe that amplifies the blocks.

If my current test works the way I think it will come out then I am going to have to start looking at each source and decide if it can take a little smoothing. I am pretty sure most sources will come out just fine and the ones that don't hopefully are less blocky to begin with.

In the end if you are looking for 1:1 quality I don't think that either XVID or X264 are there yet or may never be.
However I believe that with some more testing and tweaking I can eventually find settings I will be happy with.
Keep in mind I watch these on a 55" high def TV. Errors are noticable at that size.

Mark

[shon3i]

not true, there can be easily avc or aac codecs that are worse then mp3 or asp or mpeg2 codecsI don't know bond, how avc can be whrose than mpeg2 on same bitrate (aslo asp), i say that mpeg4 (nevermind that is asp or avc) must be better than mpeg2, or isn't, so we than not need mpeg4 anyway when is mpeg2 good enought right. I aslo say that if we have asp,avc,mpg2 for example @ 8mbs, where every codec will be trasparent, but i aslo think that avc must have about 2% better quality than others two, like avc in for example encoding @ 500kbs must have about 30+% in quality than others two. Correct me if i wrong.

Aslo AAC is supperior than mp3 about 30-60% in lower rates, and very small percent in high rates but comparing AAC @ 320kbs and mp3 @ 320, i think that aac is better because mp3 have big cutoff on 16khz.

Both AAC and AVC must be superior than other only if encoders is good.

[foxyshadis]

@Blue_MiSfit
:goodpost:

Could you tell me what kind of container you use that allows you not to transcode the ac3 or dts tracks? As far as I know, the .mp4 container does not support this.

One other thing I am testing, is the "threads option" in meGUI's profile for video encoding. By default, it is set to 1. I have a dual cpu setup, so I thought I would try to set it to 2. So far, this seems to speed things up. Comments?
Since no one answered: MKV. Maybe avi if you can find something to crowbar it in, but mkv will work without a hitch.

2 threads splits the encode into two side-by-side encodes; while it causes a minute quality loss, I've only ever seen a visible drop at the lowest, ugliest end of the bitrate spectrum.

[IgorC]

I don't know bond, how avc can be whrose than mpeg2 on same bitrate (aslo asp), i say that mpeg4 (nevermind that is asp or avc) must be better than mpeg2, or isn't, so we than not need mpeg4 anyway when is mpeg2 good enought right. I aslo say that if we have asp,avc,mpg2 for example @ 8mbs, where every codec will be trasparent, but i aslo think that avc must have about 2% better quality than others two, like avc in for example encoding @ 500kbs must have about 30+% in quality than others two. Correct me if i wrong. There is too much imprecision. First, You didn't any kind of test. We don't know if MPEG-2 is worse than ASP at such high bitrate as 8 mbps. ASP is mostly optimizied for low and middle bitrates. For high bitrates ASP ( I mean Xvid because using Divx is totally waste of bitrate due to its prefiltering for low bitrates needs) encoding it should be used sharp CM and maybe change another settings etc..
Second 8 mbps for SD or HD? It is a complex task.

Aslo AAC is supperior than mp3 about 30-60% in lower rates
Very imprecise.
How low rates are? If it's 16 kbps then AAC is superior than mp3 more than 30-60%. I think _maybe_ it's 300%.

and very small percent in high rates but comparing AAC @ 320kbs and mp3 @ 320, i think that aac is better because mp3 have big cutoff on 16khz.
It's not corect to compare AAC and MP3 at the same bitrate 320 kbps. From my limited experience with audiocodecs I think that to spot itunes AAC at 192 kbits from CD quality is more difficult than MP3 LAME free format ( > 320 kbps).
And last one. Lame mp3 has no 16 khz cut at 320 kbps. And lowpass at high frequencies has nothing to do with quality.
Itunes cuts more freqs at 192 than Lame at 320 kbps does but imo itunes (from my little limited only to me ABX test) is transparent in most of cases while Lame still has issues. Maybe fewest issues but there are still here.

[MeteorRain]

Correct me if i wrong.
hi i'm coming! (Orz

You should know this before everything: ASP, AVC, Mpeg-2, are all standards, which indicate the decoding process.

for a codec, how it encodes doesn't matter much. if the encoded bitstream can be decoded in an ASP way, it IS an ASP codec. also AVC.

if a codec, works bad, is encoding a file into an AVC-codec decodable bitstream, it is an AVC encoder.

so, all what you said, in fact, are indicating x264 is better than tmpgenc (or so) (and nero aac/winamp aac encoder than lame or whatsoever), not H.264 is better than Mpeg-2. if you'd like to compare h.264 & mpeg-2, you should put them on a technology level. for example, what different technology do they support? (h.264 works with cabac and mpeg2 not, aso)

(correct me if i'm wrong

regards
MR

[comomolo]

For the sake of clarity, I would keep it simple: it seems everyone around here accepts that x264 is the best H.264/AVC encoder and xvid the best ASP encoder. I don't want to wake up the monster by talking about "the best anything", but let's asume that for a moment if you guys agree.

When we're talking about choosing one technology or standard over another (MPEG2 vs AVC vs ASP) we're talking about using the best implementation of that tecnology out there, aren't we?

That should simplify the discussion.

[MeteorRain]

not so sure if x264 is the best avc encoder.

some ppl are thinking nero avc's better (yes, a friend of mine thinks so), and nero avc, in some extent __maybe__, works better than x264.

also, options are important. for example, the inloop-filter. lots ppl have lots opinions.

and also depends on the material. x264 doesn't work fine with blue sky, and XviD with red ball.

anyway, a good avc encoder is really better than a good asp encoder on most extents.
avc must be more and more popular in the future, w/ the development of the hardware.

[markrb]

1) no fast p skip
2) disabling the in-loop filter (for 2.2 GB rips only)
3) using M4G's high detail matrix
4) careful pre-filtering to increase compressibilty (usually removegrain mode 5 or 2, or tweaked fft3dgpu + de-banding)

1/2 DVDR is exceptional. Transparent I would submit. This is between 1/2 and 1/4 the size of the original DVD, which is excellent compression IMO.

My usual settings off the top of my head:
*all motion search options
*deblocker -2,-1
*3 bframes, all options
*6 refs
*level 1 RDO
*level 1 trellis
*no fast p skip, no dct decimation

This is a tweaked version of the HQ-Slow profile. I usually get ~5fps on my Athlon64 (see sig for details)

BTW, I usually add a tiny ammount of noise on playback with ffdshow to increase percieved detail. Luma<12, Chroma<8... Does a great job!
~MiSfit

I am trying pretty much the exact same setup with some differences minor.

Misfit could you expand on the inloop filter and what settings you tend to use for the pre-filtering. Where is the inloop filter? Are you talking about the deblocking filter inside megui (-2, -1) filter? If so I know which one, but just want to be sure.
Maybe you could post an avs file that I could customize for me if you don't mind with your typical pre-filter settings? I would love to see what they do for background blocks. As I said I have tried everything I have come across and will continue to until I am happy with the settings.

[comomolo]

not so sure if x264 is the best avc encoder.
OK, my point was: when someone refers to AVC vs ASP s/he's referring to "a very good if not the best AVC encoding implementation" vs "a very good if not the best ASP encoding implementation". I just wanted to get rid of statements like "a bad AVC encoder is worse than a good MPEG2 encoder", which are almost obvious. It's understood that anyone seeking for the best practice will hunt for the best tools too, so s/he won't use a bad encoder no matter the standard behind. That's it.

[MeteorRain]

btw, i'd more like to use constant quantization instead of two-passes processing to ensure the video is encoded under quality-based, not size-based.

cheers

[shon3i]

Very imprecise.
How low rates are? If it's 16 kbps then AAC is superior than mp3 more than 30-60%. I think _maybe_ it's 300%.
You can't compare HE-AAC and mp3, because is unfair, because mp3 not use sbr tech, but comparing mp3Pro and HE-AAC in lover rates HE-AAC is better about 30% than mp3Pro (if you have good decoder for mp3Pro)

Lame mp3 has no 16 khz cut at 320 kbps. And lowpass at high frequencies has nothing to do with quality.
Not at 16khz but have at 18khz, even is mp3 at 320 transprent, somwere on HA.org forum, someone told that (i am not sure Gabriel or Graf) mp3 can't propertly code samples even on 320kbs, So lc-aac must be anyway better than mp3 on higer bitrates. (like you say iTunes @ 192 sound very transparent than mp3 @ 320)

if you'd like to compare h.264 & mpeg-2, you should put them on a technology levelThat is not true because if i disable everything in H264 why i am then anyway use H264. Fair comparing will be to use maximum settings for H264 and maximum for mpeg2, and see what is better

[IgorC]

For a love to God. It is not 16 khz neither 18 khz.
You are wrong twice already. If you don't know then encode one sample with Lame at 320 kbps and look which lowpass it has. With lame 3.90.3 it's possible to use -k feature and keep all frequencies up to 22,050 khz. Maybe it improves something but not always. Transparency is something personal and depending on abilities and equipments of each one.

http://img110.imageshack.us/img110/9671/cmduy5.jpg
http://img110.imageshack.us/img110/4468/20khzcs9.jpg

You can't compare HE-AAC and mp3
Actually you did it first in your previous post. I simply followed your sample.

[bond]

Would you give me a hint on where, more or less, would lie the frontier between the two codecs? 1000? 3000? 6000? (kbps) (I'm always talking about x264 and xvid).there is no such general frontier. it all depends on things like compressibility of the source (the darker the easier to compress for example), resolution aso...

[shon3i]

Actually you did it first in your previous post. I simply followed your sample.I said that AAC is 30-60% better than mp3 in any case especialy in lower rates, and you say that is aac better than mp3 about 300% and quote an example

If it's 16 kbps then AAC is superior than mp3 more than 30-60%. I think _maybe_ it's 300%.

and i said that you can't compare two different codecs like that, because all codecs including LC-AAC can't defeat now HE-AAC in lower rates, but i think about 30-60% when we instead mp3 use mp3PRO, because mp3PRO sound aslo good at lower rates (not so good but about 40% whrose tahn HE-AAC)

[GodofaGap]

Can we stop now pulling percentages out of our behinds? How did you actually come up with these figures?

[Blue_MiSfit]

Since no one answered: MKV. Maybe avi if you can find something to crowbar it in, but mkv will work without a hitch.

2 threads splits the encode into two side-by-side encodes; while it causes a minute quality loss, I've only ever seen a visible drop at the lowest, ugliest end of the bitrate spectrum.

thanks for covering for me foxyshadis :)

I hadn't checked this thread in awhile, but yes MKV is the container format I prefer.

I am trying pretty much the exact same setup with some differences minor.

Misfit could you expand on the inloop filter and what settings you tend to use for the pre-filtering. Where is the inloop filter? Are you talking about the deblocking filter inside megui (-2, -1) filter? If so I know which one, but just want to be sure.
Maybe you could post an avs file that I could customize for me if you don't mind with your typical pre-filter settings? I would love to see what they do for background blocks. As I said I have tried everything I have come across and will continue to until I am happy with the settings.

Yes I mean the deblocker. It is different from (for example) deblocking in ffdshow. I dont know the details, but I do know that having it enabled tends to kill details that could otherwise be preserved at high bitrates - IMHO.

I still have some problems with the big flat gradients blocking a bit.

As far as pre-filtering is concerned, removegrain(mode=5) is basically free extra compressibility. I cannot tell the difference between the source and mode 5. mode 1 is just undot. mode 2 is a bit blurrier... I pretty much use removegrain on every encode I do.

For those that are a little noisier, I like to add fft3dgpu but this one is really no "one setting fits all". There are lots of parameters and the defaults are too aggressive for me. Reading the documentation and spending time tweaking and testing with stackvertical(source,filtered) is crucial.

I have found that at conservative settings for fft3dgpu, there tends to be a bit of banding. Gradfun2db can help these.

I used to always use undot().unfilter(-5,-5) which gives a big boost. Then I realized that this unfilter call generally softened things too much for my tastes. It's a GREAT base setting for filtering animated stuff though.

Another awesome filter that I used to use all the time but haven't recently (mainly because I forgot about it after doing a reinstall) is LRemoveDust. This is Didee's update of RemoveDust, and it's freaking cool, and fast! Search about...

Anyway, totally killing the x264 blocks in the big flat scenes is almost impossible. My theory is that if you are really looking to preserve everything with no blocking whatsoever and have the bitrate to do it (2500+, though as we know that number is totally meaningless :)) - just do XviD with 6of9 or EQM-V3HR. It doesn't have this problem in my experience.

I really thought I had this problem beat in x264, but last night I encoded Pretty Persuasion (omg... absurd freaking movie, pretty entertaining though) using only removegrain(mode =5), no deblocker, and the other settings I mentioned. I shot for 1848mb (2200mb - AC3), and when using stackvertical(source,encoded), I could not tell most frames apart. Still, in those flat areas I did notice a tiny bit of blocking. We're talking maybe 5 minutes of the whole film, but still I couldn't help staring at it. ME NO LIKE BLOCK!:eek:

I'm gonna try again with XviD tonight and see if I can murder those damned infernal blocks!

Sorry if I rambled or got OT...

~MiSfit

[shon3i]

@Blue_MiSfit nice tutorial, about this filters did you mean to use all of this filters for example

undot().unfilter(-5,-5)
RemoveGrain(mode=5)
LRemoveDust ()
etc..

or?

[Blue_MiSfit]

No... That would be too much :)

For many movies I used to do UnDot().Unfilter(-5,-5).

UnDot = RemoveGrain(mode=1), but RemoveGrain is faster (especially the SSE2/3 versions).

Now I use at least RemoveGrain(mode=5).

If I use fft3dgpu(...) I will usually follow it with RemoveGrain(mode=5), and gradfun2db(...).

I just stumbled across a PM from Didee that I got awhile back that really explained a lot to me. Here is a part of it:


Well, whatever I could tell you, for sure means SLOW processing. Not really my fault - but to get the best possible out of any source, one just cannot get away with one or two ultra-fast filters. And also not without filtering. Dedicated processing means *a lot* of computations, to get only a little effect in the end - overprocessing is evil, too. Good things always take their time. Before I would produce 5 mediocre rips in a week, I'd rather do only 1, but get a really beatiful result.

Producing average quality is not an art.


Quote:
So you think the encode would look better if I dropped pixiedust?

Probably not. Make yourself a little experiment: take one minute of the source, encode it @ quant=2 - one time *with* PixieDust(2|3), one time without any filtering. Afterwards, compare the filesizes. The conclusion is: where the Dust'ed encoding will come out with an average quant of, say, 3.2, the not-filtered encoding will need an average quant of 4.0~4.5, or something like that. See my point?

Many people say "I don't do ANY filtering, 'cause I want to stay as close to the original source as possible." Now, that's a good objection, but the wrong way to achieve it (unless one goes for VERY high bitrates, like >3000 kbps.

The point is, the encoding process will filter out very much frequencies anyways. In the end, it plays no role if frequencies were dropped by pre-filtering, or if they were dropped by the encoding process.
But: If the encoder drops frequencies, then it will drop them out of the motion-compensated error-image. And this one is a wild mixture of a) noise and b) actual image content that could not be fully compensated. Therefore, the codec will drop noise as well as it will drop actual image information. Therefore, I prefer to do careful pre-processing. There I have much more control over the distinction between noise and detail. If prefiltering is succesful in dropping much more noise than detail and/or sharpness, then the final loss in the encoding stage will be much smaller: The noise is already out, so the encoder is much less forced to drop *anything*, for a given bitrate.

And my little trick with applying "TemporalSoften(1,2,3,23,2)" before PixieDust is just for making Dust's life a little easier. The loss from that tempsoften isn't noticeable, but in fact the clip gets way calmer on a per-pixel level, so that the motion estimation engine of Dust has a cleaner source, and can work better.
Oh, and the spatial part of Dust suffers a little bit from noise-cutoff. You probably won't see it when viewing the clip, but for sure the encoder sees it during encoding. UnDot() after Dust is reasonable effective against that - even better than before it (!).

So, a better sequence than only PixieDust() alone looks like that:

TemporalSoften(1,2,3,23,2)
ConvertToYUY2().PixieDust(2).ConvertToYV12()
UnDot()

Lastly: If possible, avoid higher values for Dust than "3"!! I mostly use "2" only, and that's fully enough most times. Dust is very different from all other filters. To learn a little more about it, visit this thread, and take those few minutes for reading .


Quote:
Also, I was using avisynth's built in sharpen() function. What would YOU reccommend for a bit of sharpening when upsizing?

Short answer: iiP.

Longer answer: For sure, not a simple sharpen(). If you want a fast sharpening after dust, use unfilter(20,20)~(40,40). unfilter works only on luma, leaving chroma as de-noised as it already is, in this case, sharpen() will re-introduce a little chroma noise. And, for me unfilter does the job just a little better.

Another very good possibility is to used supersampled XSharpen, a.k.a. sharp resize:
LanczosResize(current_size * 4).XSharpen(255,255).LanczosResize(current_size)
Perhaps even with Lanczos4 instead of Lanczos. But then again, that's slow.

Yet another possibility: LimitedSharpen(). But it's also not very fast ...


It's a tad unrelated, but the thought process applies. I hope you ppl enjoy and that Didee doesn't mind :)

~MiSfit

[*.mp4 guy]

I've been working on a new matrix (http://www.yousendit.com/transfer.php?action=download&ufid=10E832D079074FD3) for "transparency", its not quite finnished yet, but It should work well If anyone here is actually aiming at transparency in X264.

My high detail matrix (http://www.yousendit.com/transfer.php?action=download&ufid=8EBC6357519773C9) has been through some changes aswell, its not quite aimed at the same area as the old one though (it is aimed at a bit higher bitrates).

Also recommended deblocking settings for both matrixes are -2:-4 for sharp results, -1:-2 for medium sharpness, and 0:1 for smooth results. I would recommend using the sharpest settings that don't casue enough artifacts to bother you.

[elguaxo]

Thanks *.mp4 guy!

[frodeste]

Since no one answered: MKV. Maybe avi if you can find something to crowbar it in, but mkv will work without a hitch.


Agreed. I will use MKV for now. I should get this to play fine in any directshow player with the correct codecs. I have been reading up on this container since I posted last.


2 threads splits the encode into two side-by-side encodes; while it causes a minute quality loss, I've only ever seen a visible drop at the lowest, ugliest end of the bitrate spectrum.

I cannot see any difference on a small display. Speed increase on the second pass was not noticeable, so I will keep it at 1 thread.

:thanks:

[markrb]

Call me stupid, but what do you mean by "transparency"?

Thanks,
Mark

[smok3]

markrb, visually indistinguishable from original = transparent (also used in audio domain, like: that mp3 is subjectively transparent to uncompressed version)

[frodeste]

I have been testing encoding a couple of DVDs to h264 using MeGUI and x264. Source: Hostage PAL DVD

AVIscript:

DGDecode_mpeg2source("V:\Hostage\VIDEO_TS\VTS_05_1.d2v",info=3)
ColorMatrix(hints=true)
#Not doing anything because the source is progressive
#crop
BicubicResize(720,384,0,0.5) # Bicubic (Neutral)
Undot() # Minimal Noise

Profile used: HQ-Insane, target bitrate: 1400 kbps without soundtrack.

Log:
Starting job job1-1 at 23:01:50
encoder commandline:
--pass 1 --bitrate 1400 --stats "V:\Hostage\VIDEO_TS\1.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-psnr --output NUL "V:\Hostage\VIDEO_TS\1.avs"
successfully started encoding
Processing ended at 00:43:26
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x384 @ 25.00 fps (162990 frames)
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [info]: slice I:1227 Avg QP:17.16 size: 31163
x264 [info]: slice P:61300 Avg QP:19.07 size: 10957
x264 [info]: slice B:100463 Avg QP:20.66 size: 4275
x264 [info]: mb I I16..4: 36.5% 0.0% 63.5%
x264 [info]: mb P I16..4: 21.9% 0.0% 0.0% P16..4: 56.2% 0.0% 0.0% 0.0% 0.0% skip:21.9%
x264 [info]: mb B I16..4: 1.4% 0.0% 0.0% B16..8: 23.4% 0.0% 0.0% direct:23.1% skip:52.0%
x264 [info]: direct mvs spatial:99.1% temporal:0.9%
x264 [info]: SSIM Mean Y:0.9819285
x264 [info]: kb/s:1398.1

encoded 162990 frames, 26.77 fps, 1399.18 kb/s

----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
job job1-1 has been processed. This job is linked to the next job: job1-2
Starting job job1-2 at 00:43:26
encoder commandline:
--pass 2 --bitrate 1400 --stats "V:\Hostage\VIDEO_TS\1.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 7 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --progress --no-psnr --output "V:\Hostage\VIDEO_TS\1.mp4" "V:\Hostage\VIDEO_TS\1.avs"
successfully started encoding
Processing ended at 11:13:57
----------------------------------------------------------------------------------------------------------

Log for job job1-2

avis [info]: 720x384 @ 25.00 fps (162990 frames)
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
mp4 [info]: initial delay 2 (scale 25)
x264 [info]: slice I:1227 Avg QP:16.25 size: 31395
x264 [info]: slice P:61300 Avg QP:17.94 size: 11095
x264 [info]: slice B:100463 Avg QP:19.38 size: 4203
x264 [info]: mb I I16..4: 22.9% 50.5% 26.6%
x264 [info]: mb P I16..4: 5.7% 10.0% 2.2% P16..4: 42.6% 16.3% 7.2% 0.6% 0.3% skip:15.0%
x264 [info]: mb B I16..4: 0.3% 0.8% 0.3% B16..8: 36.3% 2.2% 3.8% direct: 5.0% skip:51.2%
x264 [info]: 8x8 transform intra:55.5% inter:45.2%
x264 [info]: direct mvs spatial:87.6% temporal:12.4%
x264 [info]: ref P 55.2% 17.0% 8.5% 4.1% 3.0% 2.7% 2.1% 1.2% 1.0% 1.0% 0.8% 0.8% 0.7% 0.7% 0.7% 0.5%
x264 [info]: ref B 71.3% 14.3% 4.6% 2.4% 1.3% 1.1% 0.8% 0.6% 0.4% 0.4% 0.3% 0.5% 0.3% 1.2% 0.3%
x264 [info]: SSIM Mean Y:0.9863097
x264 [info]: kb/s:1400.0

encoded 162990 frames, 4.31 fps, 1401.03 kb/s

desired video bitrate of this job: 1400 kbit/s - obtained video bitrate (approximate): 1403 kbit/s
----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Found intermediate output file 'V:\Hostage\VIDEO_TS\1.stats', deleting...
Deletion succeeded.Starting job job2 at 21:08:31
encoder commandline:
-add "V:\Hostage\VIDEO_TS\1.mp4" -new "V:\Hostage\VIDEO_TS\2.mp4"
successfully started encoding
Processing ended at 21:16:53

Problem: Stutter. Several sequences play just fine, and then parts or the whole image will stutter back and forth for a few seconds. CPU usage is never above 10% during playback, and all other paramteres like RAM are well below max. Playback is now done in Nero Showtime.
The stuttering appears to be more frequent at bitrates above 1 mbit/s


I have been reading prevous posts but they don't give the clues I need. Any clues would be of great help!

Could this be because of the VBV Buffer Size?

[GodofaGap]

Well the most obvious question to ask would be if you tried another player (MPC, VLC, etc.).

[frodeste]

Well the most obvious question to ask would be if you tried another player (MPC, VLC, etc.).
Not yet, I did however try another encode using CE-Baseline. No stuttering at all.

[Blue_MiSfit]

try other players. Specifically, Media Player Classic with ffdshow handling the h.264 decoding.

[frodeste]

try other players. Specifically, Media Player Classic with ffdshow handling the h.264 decoding.
:) Already there! I will test this tonight, or sunday. But I thought I would test the CoreAVC first. Reading this forum tells me that this should be more efficient.

Thank you for your input!

[comomolo]

Call me stupid, but what do you mean by "transparency"?


markrb, visually indistinguishable from original = transparent (also used in audio domain, like: that mp3 is subjectively transparent to uncompressed version)

Well, I've been told at another thread that 3000 kbps would be "transparent". I used that bitrate and HQ-slow, but I see tons of blocks dancing around in flat colored areas. I certainly can distinguish that from the original DVD very easily.

Are we hoping for the impossible or transparency can actually be achieved?

[smok3]

comomolo, certainly not with bitrate methods, try some quality based encodings..., and even there only maybe, depends on a lot of parameters. Best bet is to do case on case method so far.

[comomolo]

Thank you smok3,

I started this thread and I think I have an answer to the posted question. From what I have grasped, all boils down to:

"Unless you have a thorough understanding of what's going on when using AVC for compressing DVDs, unless you're willing to spend hours or days tweaking the settings for each movie (effectively becoming a compressionist yourself) and until most decisions on parameters become more or less automatic (the future will tell) you're better off copying your DVDs ISOs to hard disks "as is", or reducing them in half with tools like CloneDVD. You'll need some extra space, but hard disks aren't that expensive these days..."

I guess I'm heading for a bigger disk array. In the end, when all those HD-DVD titles can be put on hard disks, I'll need even more, so I better start saving!

Thanks for a pleasant discussion and interesting points of view.

CM

[Shinjite]

:) Already there! I will test this tonight, or sunday. But I thought I would test the CoreAVC first. Reading this forum tells me that this should be more efficient.

Thank you for your input!

Recommended with CoreAVC as the decoder
FFDShow's h264 libavcodec's speed is still not up to par as what CoreAVC can achieve at the moment :)

[Morte66]

Re Mass DVD conversion for media server...

I've been down this road, to the tune of about 500 encodes so far, so here's what I think. IMHO etc:

1) If you want to build a media server for hundreds of DVDs, you've got to use some sort of MPEG4 encoder. The drive size and cost is prohibitive otherwise. MPEG2 backups are for people who burn DVD-Rs and put them in 400 disc wallets.

2) With the right setup (encoders runinng 24/7 in the background at low priority, good GUIs), slow encoders are relatively painless. They use the computer's time, rather than your time. But there is a point of diminishing returns on encoder settings -- you start getting 20% more compression for 3 times the encoder time or whatever -- so you have to pick a sweet spot.

3) As many people have said, Xvid vs X264 depends on the details of your specific material, the actual bitrate you're using, your settings, etc etc. You really have to try for yourself. FWIW, after 500 encodes with my "sweet spot" settings I ended up getting a file about half the size in double the encoding time with h264.

4) If you have a bright, contrasty LCD monitor which will reveal DGIndex and x264's block flicker problems in all their glory, you'll want to disable DGIndex's MPEG2 deblocking and use a custom x264 matrix, which will increase the file size by about 30-50%. See this thread (http://forum.doom9.org/showthread.php?t=113313) and this thread (http://forum.doom9.org/showthread.php?t=112968).

5) You don't need to hit an exact file size on a media server, that's for burning discs. So don't use a two-pass with a bitrate, use single pass constant quality mode on x264 (try the range 22 to 18) and constant quantiser on Xvid (try the range 4 to 2). This will give you faster encodes and will use a bitrate appropriate to each file (e.g. high motion action material will get more, talking heads in front of a wall will get less).

[Xvid at q2 is a special case -- it's pretty near perfect through sheer bitrate so you can turn all the clever stuff off, and thus get very fast encodes. You average about half the original size over the long term, with a huge variation per disc.]

6) Make anamorphic backups. Use the full resolution of the original DVD (720x480 for NTSC or 720x576 for PAL), and set an aspect ratio in the final mux to MKV/MP4. This way you only get one resize in the whole rip-to-view chain, during final playback. Don't let your software resize to 720x400 or whatever before encoding (lots of GUIs will try). This is a bigger deal than the fine points of encoder settings.

7) In the long run, I've averaged about 10MB/minute for 720x576 pixel DVD encodes using x264 on my crf 22 anti-blocking settings. I encode at about 7.5fps on a A64 X2 3800+. The encodes look better than the originals -- I gain far more from the Avisynth ColorMatrix filter and x264's inherent de-noising (noise desaturates colours) than I lose in the borderline-transparent compression.

8) Transparency is in the eye of the beholder. It depends on how fussy you are, how far you sit from the screen, how good your display is, and what material you're watching. When I started, I did some tests and decided that x264 at constant quality (crf) 24 was good enough for me, but I increased the quality to crf 22 to be on the safe side. After a year and several hundred encodes I bought a better monitor, and discovered that crf 24 wasn't good enough for me any more. I was very glad I'd allowed that safety margin. So think about the future...

9) FWIW, I mildly prefer MKV to MP4. It handles everything (including AC3), the muxer is faster, the files are a smidgeon smaller, and when you play it back it seems more resistant to skipping on a system that's under heavy load (useful for HD).

10) You have to make exceptions. Despite what I said about MPEG4 being the only practical solution for media servers, I still sometimes use DVD Shrink to back up to DVD-R. I've never seen any MPEG4 encoder that didn't suck (major shadow blocking) on the opening standup sketches in Seinfeld, for example.

[R3Z]

/snip

Absolutely great post mate :thanks:

I have basically came to the same conclusions you have but reading a post like yours would have saved me lots of time.

I beg any newbies to read Morte66's post several times :)

[frodeste]

Re Mass DVD conversion for media server...
...

Great! Have you created a profile in MeGUI that you can share with us? Or the commandline for encoding the video?

[Morte66]

Great! Have you created a profile in MeGUI that you can share with us? Or the commandline for encoding the video?

Here are some x264 settings based on a suggestion from Saggittaire, that are as close as I've come to "optimum bang-per-buck":

--crf 22 --bframe 2 --weightb --ref 2 --direct auto --filter -2:-2 --crf 26 --qcomp 0.75 --ipratio 1.25 --pbratio 1.33 --analyse "all" --8x8dct --me "hex" --subme 5 --progress -o

You could add an anti-blocking matrix with them if you want.


Now here are the settings I actually use. They're maybe half the speed for only 25% more compression, but they suit me fine because they encode about 3 DVDs/day, which is as fast as I want to do all the ripping and preparation etc.

Here's my MeGUI video encoder profile (http://www.joel-benford.co.uk/post/crf 22 -2-2 cqm antiblock.xml); and here's the x264 command line:

C:\Program Files\MeGUI\tools\x264\x264.exe --crf 22 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct none --filter -2,-2 --subme 6 --analyse all --8x8dct --ipratio 1.0 --threads 2 --thread-input --cqmfile "C:\Program Files\MeGUI\extra\M4G-V3.cfg" --progress --no-psnr --output "" ""

Here (http://www.joel-benford.co.uk/posts/M4G-V3.cfg) is the custom anti-blocking matrix I'm using. I find that does the job, but MP4-Guy (who wrote it) may have an improved version by now.

[akupenguin]

--crf 22 ... --crf 26
typo.

[frodeste]

..Here (http://www.joel-benford.co.uk/posts/M4G-V3.cfg) is the custom anti-blocking matrix I'm using. I find that does the job, but MP4-Guy (who wrote it) may have an improved version by now.
Interesseting.

I downloaded this "M4G High Detail 3.1" the other day from the forum (This thread). Any idea about how they compare to the one you mentioned??
I looked for an updated M4G-V3, but did not find any.

I will test your profile tomorrow with the different matrixes, after the current encoding is complete.

My test last night was okey, producing quite a small file but I think the image could be sharper. Given that I am not resizing my video, how can I set MeGUI up to give me a sharper image? In the AVIsynth file?

edit: I have read up on the subject, and see I can tweak settings in MeGUI. Great info to be read in the Forum :) Anyhow, I can see how I need more input on using AVIsynth configs also. Do you or any others add any filters to AVIsynth that improve sharpness?

[Morte66]

typo.

Yup, sorry. Only needs the 22.

[*.mp4 guy]

M4G V3 hasn't been updated, and it isn't the same thing as M4G high detail 3.1.

[Morte66]

Interesseting.

I downloaded this "M4G High Detail 3.1" the other day from the forum (This thread). Any idea about how they compare to the one you mentioned??

In technical terms, no idea. I just download stuff and keep it if I like what it does. :) I think of M4G as a stronger version of the other one -- more effective anti-blocking, at the cost of extra bitrate.

Anyhow, if you're down to two choices you shouldn't be asking me. There's a lot of personal preference in this, plus it depends on the characteristics of your monitor. Try 'em both on a couple of short clips and see what you like. Put something like "trim(5000,6000)" in your avisynth script, that'll give you about 40 seconds to test.

Given that I am not resizing my video, how can I set MeGUI up to give me a sharper image? In the AVIsynth file?

Three things you can do:

- Turn down the deblocking in x264, which you can set in MeGUI. A lot of the standard MeGUI profiles use (-2,-1) which I find soft. I like (-2,-2) personally, which seems to be towards towards the sharp end of what people at Doom9 use. You could try taking it down as far as (-3,-3), but test that thoroughly before you make a habit of it. You seem to need higher quality settings to get away with lower deblocking settings.

- Use sharpening in your player. I do this with ZoomPlayer and the ffdshow post-processor.

- Use sharpening filters in Avisynth. I don't use this, because the "preferred" sharpening will change if you change monitors and once it's in the encode you can't change your mind. Also I think you should sharpen the resized image, or use a resizer that sharpens, so you need to do it during playback. [Not everyone agrees on that last bit.]

[frodeste]

M4G V3 hasn't been updated, and it isn't the same thing as M4G high detail 3.1.
:thanks: this is good to know.

[frodeste]

... Put something like "trim(5000,6000)" in your avisynth script, that'll give you about 40 seconds to test.
Good point.
Three things you can do:

- Turn down the deblocking in x264, which you can set in MeGUI. A lot of the standard MeGUI profiles use (-2,-1) which I find soft. I like (-2,-2) personally, which seems to be towards towards the sharp end of what people at Doom9 use. You could try taking it down as far as (-3,-3), but test that thoroughly before you make a habit of it. You seem to need higher quality settings to get away with lower deblocking settings.

- Use sharpening in your player. I do this with ZoomPlayer and the ffdshow post-processor.

- Use sharpening filters in Avisynth. I don't use this, because the "preferred" sharpening will change if you change monitors and once it's in the encode you can't change your mind. Also I think you should sharpen the resized image, or use a resizer that sharpens, so you need to do it during playback. [Not everyone agrees on that last bit.]

I agree on the sharpen while playing idea, as this does not affekt the material. I have done many test now, and find that using x264.exe --qp 18 --ref 3 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -2,-1 --subme 6 --trellis 1 --analyse all --8x8dct --me umh --threads 2 --thread-input --progress --no-psnr --output gives me a great sharp transparent output, at about 1/2 the original file size. I will test some more, as I am aiming for the same quality at around 1/4 the original size. (Give or take)

Running your profile tests next.

[Morte66]

I have done many test now, and find that using x264.exe --qp 18 --ref 3 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -2,-1 --subme 6 --trellis 1 --analyse all --8x8dct --me umh --threads 2 --thread-input --progress --no-psnr --output gives me a great sharp transparent output, at about 1/2 the original file size. I will test some more, as I am aiming for the same quality at around 1/4 the original size. (Give or take)

You'll probably have to drop the crf level (say 20) to get the size down that far. There's only so much additional compression you can get with advanced settings.

If that's not satisfactory, and you stay at half the original size, then Xvid in constant quantiser mode with q=2 is a very strong alternative to x264.

Might I suggest also trying something like --filter -2,-2 or --filter -3,-2. It seems to me that turning down the x264 in-loop deblocking is a more efficient route to sharpness than turning up the overall quality level.

Running your profile tests next.

When you do, try turning off B-Frame prediction (--direct none).

Are you actually seeing flickery blocks in large dark areas? That's what the profiles will help with. If you don't have the problem, they'll just put the bitrate up for no gain.

[frodeste]

A few observations.

--crf 22 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo
--bime --weightb --direct none --filter -2,-2 --subme 6 --analyse all --8x8dct
--ipratio 1.0 --vbv-maxrate 25000 --threads 2 --thread-inputGave me 7.79 fps, 1718 kbit/s file. I like the quality.

--crf 22 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo
--bime --weightb --direct none --filter -2,-2 --subme 6 --analyse all --8x8dct
--ipratio 1.0 --vbv-maxrate 25000 --me umh --threads 2 --thread-inputGave me 6.16 fps, 1707.55 kb/s file. I can't tell the difference between the two. 7.79 to 6.16 will increase encoding time more then it is worth.

--crf 22 --ref 10 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo
--bime --weightb --direct none --filter -2,-2 --subme 6 --trellis 1 --analyse all --8x8dct
--ipratio 1.0 --vbv-maxrate 25000 --threads 2 --thread-input Gave me 5.73 fps, 1730.50 kb/s. Still can't see any major difference.

--crf 20 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo
--bime --weightb --direct none --filter -2,-2 --subme 6 --analyse all --8x8dct
--ipratio 1.0 --vbv-maxrate 25000 --threads 2 --thread-inputGave me 7.08 fps, 2414.00 kb/s. Quality is better, but not obviously so. Even when I scale the picture up a lot while playing.

All test were ran with the MP4guy's matrix as mentioned above.

Running the first test again with no matrix, produced more or less identical results regarding visual quality, at 8.00 fps, 1587.13 kb/s (as to be expected)

Running the first test again twice - with and without custom matrix - this time with directional AUTO gave 9.02 fps, 1556.62 kb/s and 8.98 fps, 1483.90 kb/s respectively.
--crf 22 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo
--bime --weightb --direct auto --filter -2,-2 --subme 6 --analyse all --8x8dct
--ipratio 1.0 --vbv-maxrate 25000 --threads 2 --thread-input Again, I am satisfied with the quality, but the filesize is significatly smaller for both tests. Is this to be expected from turning on direct?

As you can read, I went with Morte66's filter settings, with makes me happy as far as sharpness goes. It's not perfect, but acceptable given the original material and bitrate. Thanks!

I realize that one can't draw a general conclution here. I find that using the matrix will produce larger files (as expected) and in this case only minor quality changes.

My conclusion so far, is to continue using the matrix with the following options.
--crf 22 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo
--bime --weightb --direct auto --filter -2,-2 --subme 6 --analyse all --8x8dct
--ipratio 1.0 --vbv-maxrate 25000 --threads 2 --thread-input --progress --no-psnrFor this test material, I am hitting about 1500 kb/s sec. Still somewhat higher than I am looking for (around 1300+/-), but quality is more important to me the diskspace. (Up to a certain point)

What other options could I go with to lower the bitrates? Options to my AVIsynth scripts are also of interest. I am reading a lot about how to make the source material more "compressible". Important note: I don't really care how long it takes to run encoding.

Are you actually seeing flickery blocks in large dark areas? That's what the profiles will help with. If you don't have the problem, they'll just put the bitrate up for no gain. No, I do not have this problem.

[Morte66]

Your results seem pretty similar to mine, except that I'd guess my display hardware/settings are harder on blocking than yours.

What other options could I go with to lower the bitrates?

If you're not suffering blocking that bothers you, try losing the custom matrix or use the milder "high detail" matrix. Make sure you test with some scenes with steady dark backgrounds.

Options to my AVIsynth scripts are also of interest. I am reading a lot about how to make the source material more "compressible". Important note: I don't really care how long it takes to run encoding.

Running a denoiser before x264 (which does lots of denoising itself) will give you a smaller file, but they potentially remove some detail. I don't generally use them, so I'm not familiar with the newest stuff. But try putting FluxSmoothST(10,15) in your avisynth script and see what you think. If you like what it does, check out newer denoisers like RemoveGrain and FRFun (start with searches in the AviSynth Usage forum).

[akupenguin]

Again, I am satisfied with the quality, but the filesize is significatly smaller for both tests. Is this to be expected from turning on direct?
Yes. direct=none is a huge penalty, and destroys a large portion of the purpose of B-frames.
Some people dislike the direct mode decision that x264 makes in certain scenes. Completely disabling direct mode does prevent that, but with lots of collateral damage in the form of increased bitrate everywhere else.

[Manao]

--ipratio 1.0 : I can't think of any good reason to do that

--ref 5 : You could lower that down to 3 or even 2 without seeing any quality loss nor bitrate increase. The encoding time, however, will improve.

[Morte66]

Yes. direct=none is a huge penalty, and destroys a large portion of the purpose of B-frames.
Some people dislike the direct mode decision that x264 makes in certain scenes. Completely disabling direct mode does prevent that, but with lots of collateral damage in the form of increased bitrate everywhere else.

Fair point. It's part of my extreme anti-blocking approach, tailored to my display which is very revealing of blocking. Since Frodeste isn't bothered by blocking, direct=auto would be the way to go.

[Morte66]

--ipratio 1.0 : I can't think of any good reason to do that

Come to think of it, nor can I. It seems to have crept into my encoder profile. Oops.

--ref 5 : You could lower that down to 3 or even 2 without seeing any quality loss nor bitrate increase. The encoding time, however, will improve.

Well, I did get some drop in file size for crf test encodes with 5 mixed refs instead of 3. Not much, and it's not especially good value, but like I said these settings don't try to be optimal value. They're the settings which get stuff encoded about as fast as I feel like ripping it. Though I guess I did do my tests about a hundred x264 versions ago, so maybe they could use some re-checking.

[*.mp4 guy]

Yes. direct=none is a huge penalty, and destroys a large portion of the purpose of B-frames.
Some people dislike the direct mode decision that x264 makes in certain scenes. Completely disabling direct mode does prevent that, but with lots of collateral damage in the form of increased bitrate everywhere else.

Actually, the problem isn't so much that X264 makes bad decisons with direct auto, using the standard matrix. The problem is that X264 is a bit finnicky about matrixes, and the more they differ from the standard matrix, the more likely X264 is to make bad decisions using direct auto. Luckily its not as huge a compressability killer as you would think, when it is working correctly it increases efficiency by about 7.5% (in my experience). When its not working well it can cause huge "eficiency" gains, but at the cost of very noticible blocking and temporal incongruencies.

[Manao]

But you could easily scale your matrices in order to get the same size as the flat matrix. That would (mostly) avoid those bad decisions. After all, we're not in the ASP case were a 'six of nine' concept allowed to invent missing quantizers between 2 and 3. With AVC, the matrices 16,16.... 16 and 24,24... 24 behave in the same way for the rate control.

[*.mp4 guy]

The problem isn't that the matrix is a different compressability then the standard one, so much as I can tell. The problem is that the DC coeficient is very low compared to the standard matrix, or atleast thats the best idea I've been able to get of what the problem is. Anyway, the matrix in question already uses values all the way up to 255, so scaling isn't an option.

[Sharktooth]

My EQM AVC-HR matrix has unbalanced coefficients to take care of the blocking nature of the AVC codecs.
It surely may screw decisions but you gain a more crisp and non-blocky picture comparing to the standard flat quantization.
Well, it may be improved and maybe balanced not screw so much the RC, but i have no time for that (it requires a s**tload of consecutive tests).

[*.mp4 guy]

Yeah, the amount of testing required is a pain in the ass, especialy considering that linear scaling of all coeficients is something the codec should be able to do easily. Also It's very hard to test whether the compressability of a matrix is balanced when the rc can't even keep cq encodes acting normaly, you will notice that with direct mode auto my matrix has similar compressability to the standard one, the matrix was originally made before direct mode could be set to none, so as far as I could tell it was balanced. to make matters worse direct mode auto is only a problem on some sources, while it works fine on others.

[edit]
I should mention that this isn't supposed to be critisim of X264, it is after all still in alpha status, and problems like this are to be expected until X264 has been around long enough to hit 1.0.

[frodeste]

A lot of good comments guys! :thanks:

I have been reading up on the ipratio and ref settings. I have some understanding of ipratio, but not how this will change quality of the encoded material?

I see one of Sharktooth's profile changes ref to 15. Any reason for this? I find increasing this decreases the encoding speed, with I can live with if the quality is improved.

I also read a thread about increasing the number i b frames. Is this common?

I did another test with: -crf 22 --ref 10 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 1 --analyse all --8x8dct --ipratio 1.0 --vbv-maxrate 25000 --me umh --threads 2 --thread-input --cqmfile

This gave me 6.35 fps, 970.06 kb/s, with is somewhat lower framerate then what I am looking for, but the quality was ok for most parts of the move. Some scenes with stronge white light through a window in a dark room showed a lot of blocks.

What I am trying to learn here, is what effect I can expect from M.E Algorithm and Trellis. Your comments help me understand what I see when I change the different settings!

[frodeste]

I have done som more testing. Increasing or decreasing the reference frames does have som effect, but mostly on the encoding time. Nero does not like a large amount of reference frames. The source starts to stagger in certain parts of the picture.

Going from RDO to RDO level 2 does not seem to give much more quality but does increase encoding time, so I am sticking with RDO.

I have tested some with different types of ME Algorigthm, and will stick with --me umh for now.

Several testings shows me that I am getting lower bitrates for the whole movie then I get for about 40 seconds, so I have increased the quality somewhat to --crf 21 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh

I feel I am getting close to a setup that works well, gives me great quality, and encodes to a decent rate.

Just thought you might like to know.

Edit: The above settings seems to give me a bitrate in the range I was looking for, and is encoding quite quickly (~7 fps). I expect quality to come out better then for --crf 22. Which option should I test to achieve better quality? I need some help on what parameters will have the most effect (one way or the other).

[*.mp4 guy]

You could drop trellis and increase refs to 8, that would prolly put you ahead in both quality and speed, otherwise those settings are about as good as you can get right now without getting into non-objective territorry.

[frodeste]

You could drop trellis and increase refs to 8, that would prolly put you ahead in both quality and speed, otherwise those settings are about as good as you can get right now without getting into non-objective territorry.
Great! I'll try it out.

[akupenguin]

trellis is fast (except at very high bitrates). refs are slower, and ref=5 is already well into diminishing returns for most content.

[*.mp4 guy]

Ehh, sorry about that, I think I confused trellis with rdo mode 2.

[frodeste]

Okey. More testing completed.

--crf 21 --ref 8 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --sar 37:25 --cqmfile "M4G High Detail 3.1.cfg" --progress --no-psnr
encoded 162990 frames, 6.97 fps, 1146.03 kb/s

--crf 21 --ref 8 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --sar 37:25 --cqmfile "M4G High Detail 3.1.cfg" --progress --no-psnr
encoded 162990 frames, 5.87 fps, 1111.39 kb/s

--crf 21 --ref 8 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me esa --threads 2 --thread-input --sar 37:25 --cqmfile "M4G High Detail 3.1.cfg" --progress --no-psnr encoded 162990 frames, 2.66 fps, 1155.63 kb/s

--crf 21 --ref 8 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me esa --threads 2 --thread-input --sar 37:25 --cqmfile "M4G High Detail 3.1.cfg" --progress --no-psnr
encoded 162990 frames, 2.55 fps, 1111.90 kb/s

What I can conclude is:

Using Trellis from none to always slows down the encoding from ~7 to ~6 fps. It seems that the files are smaller when using trellis without effecting quality, so I am sticking with it.

I am also keeping the number of ref frames at 8, as I am getting good encodingspeeds. I don't think I have much to gain from more referance frames.

The real drop in encoding time comes from changing ME Algorithm from Multihex to exhaustive. I honestly can't see much difference between the two, so I will stick with using the Multihex setting.


Since this setting is producing close to transparent encodes for me at acceptable bitrates, I have started testing with better quality (crf) settings. My understanding is that lowering the crf value from 21 to 18, will on avarage increase bitrates and (in theory) better quality encodes. Is this correct? At least, this is what I see from my testing. I also see a drop in encoding speed.

But for the time beeing, here is my "own" personal profile setting - hopefully to help anyone starting out with x264:
x264.exe --crf 21 --ref 8 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --cqmfile "M4G High Detail 3.1.cfg" --progress --no-psnr --output "" ""

If anyone cares to comment my choice of settings, feel free.

I have spent a great deal amount of time trying to find the cross between encoding speed, avarage bitrate of resulting file, all while producing what I perceive to be a transparent encode. I think I have come close to this, with your help. So I think a :thanks: is in order.

[xyloy]

x264.exe --crf 21 --ref 8 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --cqmfile "M4G High Detail 3.1.cfg" --progress --no-psnr --output "" ""
The deblocking filter is adaptive. So 2,2 is not a good idea if you want to preserve details. And 2,2 would be at least a little meaningfull at extremly low quality(let's say a range of q40 <---> q51) on anime encodes, but not at all with --crf 18/22/26 and so on.

Besides --no-fast-pskip, --no-dct-decimate would be a good add for "transparent encoding".

You can also disable SSIM computation ( --no-ssim) if you don't use it(SSIM and PSNR computation are alike)

[nm]

The deblocking filter is adaptive. So 2,2 is not a good idea if you want to preserve details. And 2,2 would be at least a little meaningfull at extremly low quality(let's say a range of q40 <---> q51) on anime encodes, but not at all with --crf 18/22/26 and so on.
Um, he's using -2,-2, which is reasonable.

[unmei]

My understanding is that lowering the crf value from 21 to 18, will on avarage increase bitrates and (in theory) better quality encodes. Is this correct? At least, this is what I see from my testing.
Since the bitrate part is an observation and not an interpretation, it must be correct :)
Now to the interpretation:
The CRF value is similar to the quantiser in constant quantizer mode, except is not really a quantiser but sort of a "quality" parameter scaled in a way that it about represents the resulting average quantiser. Therefore higher CRF implies a higher average quantiser. And the higher the quantiser, the more data is dropped out of a block's frequency components. Even tho quality in this step is not strictly proportional to data kept, less kept frequency data usually implies less quality (when only looking at this step of course).

I also see a drop in encoding speed.
This stems (at least for a large part, i guess) from the lossless coding at the end (CABAC or CAVLC) having to compress more data.

[frodeste]

Um, he's using -2,-2, which is reasonable.
I did tests with lower settings, but found the resulting movie to not be so sharp as I like.

But point taken on both sides.

[frodeste]

Besides --no-fast-pskip, --no-dct-decimate would be a good add for "transparent encoding".

You can also disable SSIM computation ( --no-ssim) if you don't use it(SSIM and PSNR computation are alike)

I read the other thread on this with great interest. However, I can't see that turning on (Off) the SSIM in MeGUI changes the actual command line. Could this be a bug?

[dumbas..]

Well as a admitted total novice... to throw some fat on the fire...:devil: And lay myself bear to riddicle.....

This thread started by implying, without asking what is <hesitate> "best". Mpeg4 AVC vs the MPEG2 standard. For DVD's to another format.

comomolo - disagree if you feel this is an incorrect interpretation.

I have to concure in general. I have found that AVC gives me the most pleasing solution. However I retain the original DVD for future reference should (when) a better re-encoding solution appears.

Various answers appear with the 'owner' implying what they believe to be "best" for them. This is all ok by me. I am glad for the insight.
What seems to be happening is that the symantics of superior knowledge end in a battle of X264 options. Then being used in a gladiatorial contest to mark what is "best" without , of course, saying that. Because we all have bests!

My point being what is best for one (person or movie) is not best for another. There ARE no correct solutions. Please guys, remember the idiots like me who just look for help and get lost in your trading blows of --xx this --yy that.

I would love to have your in depth knowledge of the ins and outs of the codecs. I am just happy to encode and get good results.:)

DVD to avc/ac3 .mkv using Sharktooth's 'slow' profile
HD to avc/ac3 .mkv trying Sharktooth's 'SA-HD-DVD' proflie.

Thank you Sharktooth - so much!

Of course, in a sense profiles suggest a best. What I hope is that the blow trading will result in imprvement to exellent programs like MeGui. To help the less able majority , like me to get the er, best for us!

I think time to encode is a red herring. Its only an issue if your are constanlty watching it and hoping to shave a few minutes of it. A new processor and bingo- in time, we all eventually have more power. I remember pentium2 taking 6-8 to do a DVD to DIVX. Now its 18 hrs to do HD to h264. The issue is achieveing better quality at the cost of more horsepower. Running as a background on my A64 3200 I can still watch X264 re-encoded HD with MPC+Fdshow using my projector on 109"screen. The results are brilliant. Thanks, I guess to you.

File size per say, is no longer an issue. Just how tightly you clutch you wallet! CD / DVD / Hard disk it just doen to money. A new disc format and bingo, eventually we all start using that more and more. Remember 5.25 & 3.5inch diskeettes?


There are, as I see it basically two limitations at stake.

Media size. Do I use CD or DVD and shrink onto? (DVD9 source or HD stream) and hence quality constrianed.

Quality. No reasonable real size contraint (Size constraint yes, but only to the extent of the media. In hone media server terms this is almost meaning less. ).

You either vote quality or file size. Pick any one!

I started reading this thread with great interest now I'm lost and confused! I would have thought that all this grey matter should combine forces:
Use a predefined source material. ( Just don't get to lost in defining a source - as I have witnessed elsewhere!)
Do your stuff.
Produce an output.
Let the great unwashed vote.
The "winner" could the be promoted to a profile?

It would then be available to duffers like me (and I suspect 80% of the people who use this forum.

Lets harmess all this ability guys, not trade codec --option blows!

I will now hide behind the sofa and wait to be flamed...:p

[frodeste]

Why flame? :)

I see your point. My motivation was learning about the options, so I understand what I need to tweak in each movie I encode.

I now have a setting which works very well for me, and I know what to change in the "special cases". The settings I have now, I call "My profile".

One of the reasons I started searching for my own settings, was that the HQ-Insane profile, created movies with stuttering in Nero and CyberLink. Hence, quality was crap.

My goal for finding a profile was, in priority:
1. ~transparent encoding
2. ~1/4 to 1/5 disk space saving. (RAID controllers don't come cheap)
3. Reasonable encoding speed. max 12 hours.

I have that now, and am very happy. I can batch encode the video, extract chapter files and so on. The only thing I can't get to work are subtitles. But that will be another thread.

[Jay Bee]

Lets harmess all this ability guys, not trade codec --option blows!

:D
....

[dumbas..]

Why flame? :)

Not meant as a flame just a "frustration".

-I see many threads that "drop" to a trade off between posters.. that have different views of what is "best" in their view. It never seems to reach a conclusion. Folks just get tired of posting replies.

I just wanted to see an objective output of the discussion. I.E. help the idiots like me, who look for guidance to achieve improved encoding, But without the greater knowledge of people who can tweak the process.

Profiles are what make x264 tick better?

My apologies if any offence was caused, or implied. But being contentious never comes free.

I LOVE this forum. It has greatly helped me achieve backups that I would never thought achievable, and I admire its coding contributors greatly.

[frodeste]

Not meant as a flame just a "frustration".

-I see many threads that "drop" to a trade off between posters.. that have different views of what is "best" in their view. It never seems to reach a conclusion. Folks just get tired of posting replies.

I just wanted to see an objective output of the discussion. I.E. help the idiots like me, who look for guidance to achieve improved encoding, But without the greater knowledge of people who can tweak the process.

Profiles are what make x264 tick better?

My apologies if any offence was caused, or implied. But being contentious never comes free.

I LOVE this forum. It has greatly helped me achieve backups that I would never thought achievable, and I admire its coding contributors greatly.

I did understand you, no offence caused to me :)

I also agree on your posting, hence I said why flame. But I do like this discussion.

An objective what's best is hard to do. That is what I was aiming for in my tests. On the subjecive side, well that's another matter.

[Thunderbolt8]

7) In the long run, I've averaged about 10MB/minute for 720x576 pixel DVD encodes using x264 on my crf 22 anti-blocking settings. I encode at about 7.5fps on a A64 X2 3800+. The encodes look better than the originals -- I gain far more from the Avisynth ColorMatrix filter and x264's inherent de-noising (noise desaturates colours) than I lose in the borderline-transparent compression.

do you mean the encoding time with 10mb/minute or the output file size ? if the latter, the one of the whole container, or only of the video stream ?
I used the same command line you posted there + same matrix (apart from the denoising thing, couldnt find where you configured that), but my filesize for Dr.No for example (105 mins, PAL 720x576 clever anamorph encoding with encode non-mode 16) is like 1,5 GB large (videostream only), although it should be like 1050 MB according to your statement. what am I doing wrong ?
where to find that inherent de-noising (noise desaturates colours) btw ?

[frodeste]

do you mean the encoding time with 10mb/minute or the output file size ? if the latter, the one of the whole container, or only of the video stream ?
I used the same command line you posted there + same matrix (apart from the denoising thing, couldnt find where you configured that), but my filesize for Dr.No for example (105 mins, PAL 720x576 clever anamorph encoding with encode non-mode 16) is like 1,5 GB large (videostream only), although it should be like 1050 MB according to your statement. what am I doing wrong ?
where to find that inherent de-noising (noise desaturates colours) btw ?

Filesize will vary because you are encoding using a "fixed quality" setting.
In some cases, I get around 1100 in bitrate, in others I get 1500, all depending on the material.

The denoising can be done in your avisynth script.

[Thunderbolt8]

Filesize will vary because you are encoding using a "fixed quality" setting.
In some cases, I get around 1100 in bitrate, in others I get 1500, all depending on the material.

The denoising can be done in your avisynth script.
im newb on that field, so could you plz elaborate which one and how to modify it ? it is really the same morte66 meant, since he spoke from the x264 "inherent" denoising, so not sure if he meant the avisynth script ?

I also used your latest posted settings here (x264.exe --crf 21 --ref 8 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-2 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --cqmfile "M4G High Detail 3.1.cfg" --progress --no-psnr --output "" "") only difference is the M4G-V3 matrix instead the high detail one (couldnt find it)
and just the videofile itself (again dr.no, 105 mins) is even higher than before with morte66's settings, its 1,72 GB. is that somehow comparable to your outcome of comparable files ? seems rather high to me.

how does constant quality work at all, or better said why does/should it give a better quality than normal 2 pass with lets say HQ-slowest options ? is the bitrate determined by the highets frame in the file and therefore used for all other frames ? if not, isnt the system of 2 pass with variable adjusting (high motion pictures get more, no action (or such) pictures get lower bitrates).
if constant quality is really better quality wise, I'd like to keep it, but I want to know if theres a way to estimate the outcoming file size (of the video stream). I'd like to stick to the max number of 1.46 GB for a 2 cd file (=730 mb for one), so that you could actually burn 3x2 cd movies onto one dvd-r (which is what i need to do all the time). but it seems hard to have control about that in constant quality mode with its -crf options, where its said the filesize should have about XX outpute size, but when I test it myself its like 500 MB more. quite unpredictable.

btw.another minor problem, seems like I cant get a proper fourCC info with x264. I chose that in x264 config of MeGUI and also typed that at the video options in the muxer (mkvtoolnix), but when I open the file with virtualdub(mod) I get (apart from quiete some error message about the video stream) a unknown fourCC message, it says just yyyy (uncomressed rb80) or something like that. actually as long as it does not harm the quality of the video its not that bad, but I just like to know why thats happening >_>

[foxyshadis]

Const. Quality won't give a better quality than two pass, but at exactly the same bitrate it'll be approximately the same quality. (With minor differences like direct auto.) But it's faster and it guarantees a certain quality; if you guess wrong in 2-pass you have to re-encode. If you must have it fit a certain size, though, then you'd use 2-pass instead. btw, megui will let you calculate fractions of a DVD, so you can use the bitrate calculator with that instead.

VDubMod requires specific hacks to be made to the file in order to parse it, which make editing problematic, among other things; MeGUI won't supply the hacks.

[Thunderbolt8]

if you guess wrong in 2-pass you have to re-encode.
what exactly do you mean with that, mean I guess the entered bitrate wrong = too low ?
or do you mean other options, in that case I just stick to the hq-slowest profile, maybe with only differences to lower the reference frames to 8 and use a custom matrix.