Scene changes not detected correctly when encoding in x264 vfw codec

[cesm23]

It's like this, for several months i have been encoding tv captures i do, to xvid. And i always get good scene changes keyframes at the correct points with xvid codec, i need this because i will do a lot of cutting in the video afterwards loslessly (in vdub) and that's why i need precise scene change detection. I did this with xvid's default settings.

Now i am using x264 vfw codec in vdub, and also using the default settings, but NONE of the scene changes are getting detected, the i-frames are getting included pretty much in random places and never in the scene changes at all... I already did a lot of tweaking in the codec's settings, and it seems that it's something to do with the motion stimulation settings, since when i changing the following settings i got a lot better scene changes detection :

- changing partition decision to 5 or lower
or
- setting range to 9 ou 10
or
- setting method to another than umh

Altough even like that i still get a few scene changes i-frames not detected correctly, at least on xvid those get detected correctly but the big problem is that by touching these settings i am afraid to be losing quality or introducing artifacts in the video which i would only discover months later, so i don't know which of those settings is "safer" to touch.

Besides i already read the x264 vfw settings guide but just imagine if i have to MANUALLY tweak the values for every video i encode... yes because i noticed that the settings i touched above seems to work better in some videos and worse in others ! If xvid is able to do the scene change correctly in ANY video with the default settings, it's not normal that x264 vfw codec is not, there has to be some equivalent settings in x264 to xvid to achive this, because if not, and if i have to manually tweak the settings for every single video to detect the scene changes correctly, just FORGET it i will be forced to go back to xvid...

Also obviously using megui or non-vfw x264 encoders is out of the question because these videos will need tons of cutting and even joining all in direct stream copy... I mostly switched to x264 thanks to the deblocking feature and better quality when comparing with xvid even though this is the vfw version of x264, but it's not possible to set a universal scene change detection settings like there is in xvid, i will really have to go back to xvid.

[Dark Shikari]

1. Scene change detection is not designed to detect scenecuts explicitly. Rather, it's designed to optimize I-frame placement. The optimal placement may not be on actual scenecuts.

2. If you're using an old x264 version (you probably are) and unthreaded encoding, you're using the old scenecut algorithm, which IMO is not very good. I removed it a month or two ago; the (now-obligatory) lookahead scenecut is completely insensitive to practically all encoding settings so it won't get tweaked by the settings you choose.

3. If you need scenecuts at specific points, you can use the qpfile option to specify a list of frames at which you need scenecuts. This is the correct way to do it.

4. I'm pretty sure you're confused, because everything you listed with regards to splicing/etc can be done more easily without VFW.

[cesm23]

1. Scene change detection is not designed to detect scenecuts explicitly. Rather, it's designed to optimize I-frame placement. The optimal placement may not be on actual scenecuts.


Hum... you mean in this x264 vfw version or in all codecs in general ? I say this because at least with xvid default settings i always got almost perfect scene detection, it was very rare to have a i frame which wasn't in a scene change (note that i found this on hundreds, if not thousands of tv captured files), just like i need more but i would prefer this in x264 too.

2. If you're using an old x264 version (you probably are) and unthreaded encoding, you're using the old scenecut algorithm, which IMO is not very good. I removed it a month or two ago; the (now-obligatory) lookahead scenecut is completely insensitive to practically all encoding settings so it won't get tweaked by the settings you choose.

I am using the lastest version that came out five days ago in sourceforge x264vfw project, version 17_1145bm_18747. does this release has that same problem you say ? (by the way i only have a single cpu core)

3. If you need scenecuts at specific points, you can use the qpfile option to specify a list of frames at which you need scenecuts. This is the correct way to do it.

Correct way ? You mean, in x264... even if in xvid it wasn't the correct way it was near perfect detecting almost all scene changes and i only need to use the default settings... Well then even in the non vfw x264 encoders this is the only way of having the correctly scene cuts, by indicating manually all of them?

4. I'm pretty sure you're confused, because everything you listed with regards to splicing/etc can be done more easily without VFW.

Splicing ?? You mean, being able to cut and join mkv h264 files without recoding ?? How ?? Yes because that one is very important to me, because without this the video will have to be recoded about 2-3 more times, and also waste time in those encodings, unless there is a easy way to do direct stream copy editing in normal h264 encoded mkv files.

The only way i know for now is demuxing the avc raw stream and then using avc2avi and then editing the video in virtual dub, is this what you mean by easier?

[MasterNobody]

the i-frames are getting included pretty much in random places and never in the scene changes at all...How did you check this? Did you take into account 1-2 frames VfW-decoder lag due the B-frames (for which there is no hack to fix it as for encoder lag in VirtualDub, and this is the biggest problem of VfW)?

[cesm23]

I know that because i VERY frequently navigate in virtual dub by keyframes (GO - NEXT KEYFRAME or GO - PREVIEWS KEYFRAME) to cut videos losslessly, so i never had such problem (that of scene changes not being detected) in xvid encodings, only now with x264 vfw encodes, the places where most scene changes happened, in the xvid encoded version it's about 95% of the times in the correct places, while in x264 vfw encoded version it mostly only inserts key frames each 250 frames, very rarely they happen at scene changes, which means that dark shikari is correct about not being meant to detect scene changes but ONLY at x264, don't make me believe that xvid's scene change detection is a LOT better than x264's... besides it's ridiculous having to manually indicate the scene changes in x264 encodings while in xvid it's already automated and works about 95% of the time, i would waste tons of time in this.


How did you check this? Did you take into account 1-2 frames VfW-decoder lag due the B-frames (for which there is no hack to fix it as for encoder lag in VirtualDub, and this is the biggest problem of VfW)?

Wait a minute... now that you talk about this i think i know what you mean... but strangely, that does only happens in videos i have not encoded myself, all the videos i encode in xvid never display this problem in virtual dub (i only do 1 pass encodings at a fixed bitrate because i do way too many tv captures to be using 2pass encodings specially with a single core cpu). Anyway the example i said in the topic was in my own encodings, so if we are tallking about the same thing, it's not taking that into account then.

[Dark Shikari]

Hum... you mean in this x264 vfw version or in all codecs in general ? I say this because at least with xvid default settings i always got almost perfect scene detection, it was very rare to have a i frame which wasn't in a scene change (note that i found this on hundreds, if not thousands of tv captured files), just like i need more but i would prefer this in x264 too.In codecs in general, but particularly in x264. Note that x264's lookahead scenecut (now the only option) is much like Xvid's and should give very similar results in practice.Correct way ? You mean, in x264... even if in xvid it wasn't the correct way it was near perfect detecting almost all scene changes and i only need to use the default settings... Well then even in the non vfw x264 encoders this is the only way of having the correctly scene cuts, by indicating manually all of them?No, what I'm saying is that if you need scene changes at a few specific points, such as to indicate chapter points, you cannot rely on the encoder to get them all for you.

If your scenecuts are mostly 250 frames (default gop size) apart, you can try raising the scenecut threshold beyond the default 40. If they're all 250 frames apart, perhaps you accidentally turned off scenecut detection, or x264vfw is (unsurprisingly!) broken.Splicing ?? You mean, being able to cut and join mkv h264 files without recoding ?? How ?? Yes because that one is very important to me, because without this the video will have to be recoded about 2-3 more times, and also waste time in those encodings, unless there is a easy way to do direct stream copy editing in normal h264 encoded mkv files.

The only way i know for now is demuxing the avc raw stream and then using avc2avi and then editing the video in virtual dub, is this what you mean by easier?Huh? You can cut and join MKV files with MKVmerge without re-encoding.

[cesm23]

In codecs in general, but particularly in x264. Note that x264's lookahead scenecut (now the only option) is much like Xvid's and should give very similar results in practice.

"lookahead" scenecut ? Is that the name of a option in x264 settings ? (sorry if i am not much into x264 encoding i am just starting in x264 encoding) And also you mean this scenecut being like xvid's, as the one in the vfw version of x264 i mentioned OR only in non vfw x264 encoders?

No, what I'm saying is that if you need scene changes at a few specific points, such as to indicate chapter points, you cannot rely on the encoder to get them all for you.

Well... usually that's what i always did in xvid i usually always get keyframes in the scene changes, very rarely i needed to use smart rendering in vdub in the very few cases there was not a keyframe in the scene change i wanted. Anyways yes i got your point on that now.

If your scenecuts are mostly 250 frames (default gop size) apart, you can try raising the scenecut threshold beyond the default 40. If they're all 250 frames apart, perhaps you accidentally turned off scenecut detection, or x264vfw is (unsurprisingly!) broken.

"unsurprisingly" looool Well.. maybe that's true because i even used the value 100 and i noticed no change at all, BUT i did noticed a MUCH better detection when i tweaked the motion stimulation settings, is this normal ? Even tough i still got more missing keyframes than in the xvid encoding.

Huh? You can cut and join MKV files with MKVmerge without re-encoding.

WHAT?? No kidding? That's very good of course i didn't knew that... is this really true even in mkv h264 encoded files ?? I am going to try it right away, because i talked to a lot of people that encode a lot in x264 and also googled a lot and was unable to find anywhere a way to cut and join h264 video encoded files without doing that demuxing... that's strange, is this a new recent feature of mkvmerge?

By the way by splitting files, i mean, by choosing a frame range, not splitting the mkv in segments of a nr of mbs each...

[Dark Shikari]

"lookahead" scenecut ? Is that the name of a option in x264 settings ? (sorry if i am not much into x264 encoding i am just starting in x264 encoding) And also you mean this scenecut being like xvid's, as the one in the vfw version of x264 i mentioned OR only in non vfw x264 encoders?It's in all versions of x264. Lookahead refers to the fact that it runs ahead of encoding (as Xvid's does) rather than the old x264 scenecut (which ran after encoding a frame; if it found a scenecut, it would re-encode the frame as an I-frame)."unsurprisingly" looool Well.. maybe that's true because i even used the value 100 and i noticed no change at all, BUT i did noticed a MUCH better detection when i tweaked the motion stimulation settings, is this normal ? Even tough i still got more missing keyframes than in the xvid encoding.I suspect something is very, very broken if changing the scenecut threshold does nothing. Motion search should have absolutely no effect on scenecut because the lookahead doesn't use anything above hex.WHAT?? No kidding? That's very good of course i didn't knew that... is this really true even in mkv h264 encoded files ?? I am going to try it right away, because i talked to a lot of people that encode a lot in x264 and also googled a lot and was unable to find anywhere a way to cut and join h264 video encoded files without doing that demuxing... that's strange, is this a new recent feature of mkvmerge?Nope, been around for ages :D

[cesm23]

It's in all versions of x264. Lookahead refers to the fact that it runs ahead of encoding (as Xvid's does) rather than the old x264 scenecut (which ran after encoding a frame; if it found a scenecut, it would re-encode the frame as an I-frame).

Well then i will have to try a non vfw encoder, because clearly this vfw x264 encoder's scene detection is a LOT worse than xvid's, or maybe it's indeed broken.

I suspect something is very, very broken if changing the scenecut threshold does nothing. Motion search should have absolutely no effect on scenecut because the lookahead doesn't use anything above hex.

Indeed i also find it strange... by the way i am using ABR quantizer encoding method, at 1500kbps (1 pass encoding), i presume this has no effect in the effectiveness of the scene change detection.

About the mkvmerge well... this way of splitting files is A LOT harder without a visual editor... besides i would need to cut by frames numbers and not time settings, unless i have to investigate better if it's possible to change a specific frame range to which mkvmerge splits the time ?

And also if it's possible to use smart rendering in cases there isnt a keyframe in the place i cut... because without these things how can this possibly be easier than using vdub ?

Anyway still thanks a lot for your help on this.

Indeed it would be great that i could make finally the transition from vfw to non-vfw but as you can see i am finding a lot of trouble for being able to cut video as fast and easy as i usually do in vdub (i even use only keyboard), i hope there's a workaroung this...

[Dark Shikari]

About the mkvmerge well... this way of splitting files is A LOT harder without a visual editor... besides i would need to cut by frames numbers and not time settings, unless i have to investigate better if it's possible to change a specific frame range to which mkvmerge splits the time ?AFAIK it can split by both frame numbers and times. In either case, it'll split on the nearest I-frame.And also if it's possible to use smart rendering in cases there isnt a keyframe in the place i cut... because without these things how can this possibly be easier than using vdub ?See above.

[cesm23]

ok thanks i will check that out.

update : that's odd i can't find afaik in google... is this really a sotware to split mkv files ? (never heard of afaik before lool)

update2 : well in fact i now tested this better in x264 vfw and the scenecut threeshold DOES have a effect but like even at 100 it still does worse than xvid, how can this be possible...

[JEEB]

AFAIK - As Far As I Know, the app is called mkvmergegui (mmg.exe) in a package called mkvtoolnix.

As for cutting the stream you'll be encoding... Avisynth's Trim(<beginningframe>,<endframe>) comes to mind with AvsP (or VirtualDub, whichever you prefer for finding your frames for cutting / preview) as you mention 'keyboard-only'. That way you could just let the actual x264 encoder eat the avisynth script as input and encode it like that.

[cesm23]

AFAIK - As Far As I Know, the app is called mkvmergegui (mmg.exe) in a package called mkvtoolnix.

Lool jeez i still have to learn more english expressions, but this is normal since i am portuguese and clearly my native language isn't english.

As for cutting the stream you'll be encoding... Avisynth's Trim(<beginningframe>,<endframe>) comes to mind with AvsP (or VirtualDub, whichever you prefer for finding your frames for cutting / preview) as you mention 'keyboard-only'. That way you could just let the actual x264 encoder eat the avisynth script as input and encode it like that.

well... that is a good idea but this way i presume it will use re-encoding and what i would need is a kind of lossless editing like vdub does in direct stream copy, because in virtual dub using the keyboard i do file splits in less than 5s while this way with mkvmerge it will take a LOT longer. I will have to explore mkvmerge better, since i am unable to find there the option to split by frame numbers, and i just downloaded the latest version 2.8.0. (and yes i am using mmg.exe)

[JEEB]

A) Avisynth is a frameserver. You give it commands, it gives you frames, you will use it for the input to the encoder. No need for lossless passes if you don't do heavy filtering on the source.

Example:

Mpeg2Source("hoge.d2v", cpu=0) #You can also use AviSource(), ffvideosource() or whatever your source can be opened with
Trim(0, 1500)++Trim(3200,12000)++Trim(16000,0)
#You get frames 0-1500,3200-12000,16000-endframe all decoded from the source, sticked together and given out

B) mkvmergegui clips by timecodes. You can mostly get those from the app you're previewing the clip in (Virtualdub and AvsP both handle this). You use this app to clip the clip you've already encoded into H.264.

C) Welcome to the interwebs.

[cesm23]

ok thanks for the code, meanwhile i would still like to solve the problem of the scenecut in x264 latest vfw codec. I think i will use 100 as the threshold for now, and use smart rendering in the places there isn't a needed keyframe, i don't see other way since i am definetly not prepared yet to jump to non-vfw based video editing, for several reasons i don't need explain here... (i would have to explain exactly all the types of cuts i need in videos and the type of tv shows i capture)

[Dark Shikari]

ok thanks for the code, meanwhile i would still like to solve the problem of the scenecut in x264 latest vfw codec. I think i will use 100 as the threshold for now, and use smart rendering in the places there isn't a needed keyframe, i don't see other way since i am definetly not prepared yet to jump to non-vfw based video editing, for several reasons i don't need explain here... (i would have to explain exactly all the types of cuts i need in videos and the type of tv shows i capture)How about you test encoding the normal way to make sure that the VFW version isn't broken?

[cesm23]

The normal way ? you mean using all the default settings of the codec ? But that's exactly what i have been doing all this time, that's why i made this topic in the first place, because only yesterday i discovered this, now i notice that ALL the encodings i did in the last weeks and months using x264 codec have this problem, while in the hundreds of xvid encodes not a single one has that problem. right now i tested the scene cut threeshold at 100 and like i said it's not broken after all, but it still isn't as good as in xvid, this is not normal...

Can i upload a small video sample ? I don't post much here so i am not exactly aware if i can post a video sample, because it's a tvcapture from a tvshow... If i uploaded it, if you would encode it to xvid, and then opening it again to encode to x264 (vfw version) and you would notice a huge difference in the places where the keyframes get inserted.

(just remember the version of the x264 vfw encoder i am using)

[Dark Shikari]

The normal way ? you mean using all the default settings of the codec ?No, meaning not VFW.

[cesm23]

oh... i see. ok i will try it and write some feedback soon. I will be using latest version of megui by the way and the x264 abr quantizer mode at 1 pass only like i always do.

[LoRd_MuldeR]

I will be using latest version of megui by the way and the x264 abr quantizer mode at 1 pass only like i always do.

Aren't "ABR" and "(Constant) Quantizer" two contradictory encoding modes? :p

[cesm23]

Aren't "ABR" and "(Constant) Quantizer" two contradictory encoding modes?

oops lol i confused the names, since i was only saying that from memory... In megui it's just called ABR, and in the vfw version it's called "single pass - bitrate based (ABR)".

Now regarding the test i did :

oh, i knew it... the keyframes get detected exactly at the same place as the vfw x264 codec (at least with the default setting of 40 at the threeshold). For example i did like this :

I used a 01:23m video which when we watch we see 29 scene changes (note that i am using my eyes to manually detect the scene changes, not the keyframes of the encoded video itself).

Now when i encode it with xvid, i get ALL the scene changes perfetly detected, there isn't a single keyframe that isn't in a scene change, just like i need (altough it's normal that in other videos sometimes there is a few missing when the scene change has very similar frames or it's a fading part).

When i encoded both in x264 vfw and megui's x264 encoder (i presume it's what you call "normal way") both with their default scene change settings, well... there is only 13 scene changes detected in both versions ! In the mkv version i encoded in megui, to know this i used mkvmerge to split in files of 1s, so that it will only split at keyframes, and the places that each of the 13 files begin is the same as the x264 vfw version inserted the keyframes (of course each splitted file hasn't 1s), so both versions use the same way of detecting scene changes.

Then i did the same as above but using the scene detection at 100 in both the vfw version and in the megui version, and now i got 22 keyframes in vfw version and 17 split files in megui... But a few of them were incorrectly detected probably because of the huge scene detection threeshold so clearly this setting isn't the solution, since even like this it missing important scene changes that xvid were able to detect.

Is this normal ? I mean if the x264 scene detection part is similar to xvid as i understood why this gives such different results, there is something wrong here...

[Dark Shikari]

Post a sample of the encoded .h264 file.

[cesm23]

Ok, since it's the first time i do this here in the forum... can i use a host like megaupload and post the link here? (the video file has 15mb)

And also if i should post the version i encoded with the default setting of 40 at the threeshold or not.

Oh i almost forget... mp4 or mkv version ?

[LoRd_MuldeR]

oops lol i confused the names, since i was only saying that from memory... In megui it's just called ABR, and in the vfw version it's called "single pass - bitrate based (ABR)".

That's the worst quality encoding mode of all the encoding modes supported by x264 :scared:

I highly recommend you use the "Two Pass" mode, if you need to hit a specific (average) bitrate. If you need to do it one single pass, then use CRF...

[Chengbin]

Ok, since it's the first time i do this here in the forum... can i use a host like megaupload and post the link here? (the video file has 15mb)

And also if i should post the version i encoded with the default setting of 40 at the threeshold or not.

Oh i almost forget... mp4 or mkv version ?

Megaupload is fine. Container doesn't matter, so mp4 or mkv is both fine.

[cesm23]

Yes but i am still very unexperienced in using megui, so far i mostly used virtual dub and pegasys xpress tmpeg only, and usually i don't understand much about 2 pass encoding yet, but won't that take TWICE the time it takes to encode at this method i say ? Using the method i use to encode it takes TWO hours (because the avi is being encoded at 720x576 1500kbps in x264 vfw version) to encode... anyway i have more experience in xvid 1 pass encoding i still need a lot of pratice to be able to use 2pass encoding correctly and to know which settings to use, unless you tell me how do i know right way which are the best settings for 2 pass encoding in x264 :) By the way watch the video sample and tell me if still looks that bad because of using this encode or not... lol

Ok it's the link to the sample video. Please notice a few things :

http://www.megaupload.com/?d=0PXTGD6N

- the original captured video, was captured by hardware tv capture card at 720x576 mpeg2, which i don't have anymore, then i converted the mpeg2 file to a avi file with the same resolution, 1500kbps bitrate 1 pass abr and applying the deinterlace filter (interpolation - adaptive, i used the pegasys xpress tmpeg sofwtare, latest version).

- then using virtual dub in direct stream copy i cut a few parts of that encoded video, because it's the huge 3d animated letters that i need the keyframes being detected so no need to include the entire live action segments with the actors, then i exported that to a avi file in direct stream copy

- that's the file i encoded in megui using all default settings, and the ABR encoding mode at 1500kbps

See those parts with the huge 3d letters ? My goal is to be able to strange the live action segments with actors WITHOUT having the 3d letters on them in direct stream copy. If you encode this mkv to xvid using all default settings like i did and jump by keyframes in virtual dub you will see that all the parts with the 3d animated letters will have keyframes, while in the x264 megui version only a few are caught, so please try comparing the keyframes locations with a xvid version of this mkv and notice that most of the 29 scene changes are missing in the x264 file.

By the way, VERY important, please someone tell me if the video is also well deinterlaced or not, just to be sure :)

[LoRd_MuldeR]

Yes, a 2-Pass encode roughly takes twice the time of a Single-Pass encode. But it's definitely worth the effort!

And as said before: If you don't need to hit a specific bitrate, you will be far better off by using a Single-Pass "CRF" encode (only Single-Pass "ABR" is bad).

BTW: With VfW setting up a 2-Pass encode is pain. A tool like MeGUI or Avidemux can do it in an automated way...

[Dark Shikari]

x264 - core 67 r1145M d2e1e1c - H.264/MPEG-4 AVC codec - Copyleft 2003-2009 -
http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 ana
lyse=0x1:0x111 me=hex subme=6 psy_rd=1.0:0.0 mixed_ref=0 me_range=16 chroma_m
e=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 chroma_qp_offset=-2 threads=1 nr=
0 decimate=1 mbaff=0 bframes=0 keyint=250 keyint_min=25 scenecut=40 rc=abr bi
trate=1500 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00
Your encoding settings suck terribly. But ignoring that.

Frame 60: not a scenecut, because the encoder can re-use the blue background even though the middle letters changed.

Frame 120: Same.

Frame 251: Not a scenecut, because it's a zoom-in through one of the letters, so there's no single frame at which it's worthwhile to force all-intra.

Frame 287: Not a scenecut, because x264 strongly biases against having two scenecuts in a row (also see min-keyint).

Frame 334: Not a scenecut, because the encoder can reuse the bottom area of the frame (the scrolling ticker), which due to the sharp lines on the text uses up a lot of bits.

Frame 402: Same.

(etc)

[cesm23]

Yes, a 2-Pass encode roughly takes twice the time of a Single-Pass encode. But it's definitely worth the effort!

yes of course i know it's worth but... i usually have 3-4 shows like this, about 40-50 minutes each to encode daily, do you really think i would let the pc encode hours and hours non stop for this ? :) I usually can only have encodings run at night while i am not using the pc, and with 2 pass encodings it would take forever specially for a single cpu core like the amd athlon 64 3200 (2ghz) i have (and unfortunately i can't afford a new cpu and motherboard, there are a lot better cpus for my motherboard available but they don't exist in the pllace i live, they now all have a am2 socket and mine is still a 939 socket).

And as said before: If you don't need to hit a specific bitrate, you will be far better off by using a Single-Pass "CRF" encode (only Single-Pass "ABR is bad).

CRF ? Hum... the problem is knowing which is the best value for me, and usually everytime i tried other methods like this in 1pass encoding, when i tryed to make them have the same filesize as the one in ABR, it would ALWAYS have worse quality... But i can try again, because you see i can't have bigger filesizes than the ones i get using abr... I will try again the crf mode, while trying to maintain more or less the same filesize that abr has with the bitrate i use.

Oh by the way, in case i need to use re-coding, which is better, abr or crf ? Can crf lose less quality in the case i need to recode in x264 ?

Your encoding settings suck terribly. But ignoring that.

Lool yes i know that's absolutly obvious, i already said i understand almost nothing about x264 encoding, this is in fact some of the first encodings i am doing in x264 using a non-vfw version... but i am sure that to learn how to use x264 settings in the best way will take weeks of pratice right ? the problem is that i don't have time at the moment to study in depth all x264 settings, and unfortunately i need to solve this problem as soon as possible since i am runnig out of hard drive space so i can't keep the mpeg2 original video files this is why i need to convert them.

Regarding the frames you mentioned, the ones i absolutly need for direct stream copy are 251, 467, 828, 1286, 1566, etc, lets suppose i want to get only the live action segments WITHOUT the 3d letters included in the segment, using direct stream copy to avoid recoding the whole segment, and not having to manually find their frame numbers. Usually when encoding in xvid i get only the segments without any recording, but in x264 they aren't there for the reasons you mention in your post, so how am i supposed to get the segments in direct stream copy without the 3d letters on them ? Yes, that's right... need to recode the whole segment, while in xvid this isnt necessary, i don't see how is this better...

Ok just tell me what you would do in my place... you would recode the entire segments just to remove the 3d letters ? Or search trough the whole episode for all the positions of the 3d leters beginnings and endings to indicate as scene cuts ? The problem is that like this not only is a lot slower than in direct stream copy, but also later i will use filters on these segments to move the damned scrolling tickler or at least to cover the letters with a blue color, i don't do this right now since i can't keep the 3,4gb original captured files right now, and i don't have time to be tweaking such filters at the moment, that's why i would like to avoid recoding...

Jeez probably i will still be forced to go back to xvid, but i wouldn't really like that...

Yes, a 2-Pass encode roughly takes twice the time of a Single-Pass encode. But it's definitely worth the effort!

yes of course i know it's worth but... i usually have 3-4 shows like this, about 40-50 minutes each to encode daily, do you really think i would let the pc encode hours and hours non stop for this ? :) I usually can only have encodings run at night while i am not using the pc, and with 2 pass encodings it would take forever specially for a single cpu core like the amd athlon 64 3200 (2ghz) i have (and unfortunately i can't afford a new cpu and motherboard, there are a lot better cpus for my motherboard available but they don't exist in the pllace i live, they now all have a am2 socket and mine is still a 939 socket).

And as said before: If you don't need to hit a specific bitrate, you will be far better off by using a Single-Pass "CRF" encode (only Single-Pass "ABR is bad).

CRF ? Hum... the problem is knowing which is the best value for me, and usually everytime i tried other methods like this in 1pass encoding, when i tryed to make them have the same filesize as the one in ABR, it would ALWAYS have worse quality... But i can try again, because you see i can't have bigger filesizes than the ones i get using abr... I will try again the crf mode, while trying to maintain more or less the same filesize that abr has with the bitrate i use.

Oh by the way, in case i need to use re-coding, which is better, abr or crf ? Can crf lose less quality in the case i need to recode in x264 ?

Your encoding settings suck terribly. But ignoring that.

Lool yes i know that's absolutly obvious, i already said i understand almost nothing about x264 encoding, this is in fact some of the first encodings i am doing in x264 using a non-vfw version... but i am sure that to learn how to use x264 settings in the best way will take weeks of pratice right ? the problem is that i don't have time at the moment to study in depth all x264 settings, and unfortunately i need to solve this problem as soon as possible since i am runnig out of hard drive space so i can't keep the mpeg2 original video files this is why i need to convert them.

Regarding the frames you mentioned, the ones i absolutly need for direct stream copy are 251, 467, 828, 1286, 1566, etc, lets suppose i want to get only the live action segments WITHOUT the 3d letters included in the segment, using direct stream copy to avoid recoding the whole segment, and not having to manually find their frame numbers. Usually when encoding in xvid i get only the segments without any recording, but in x264 they aren't there for the reasons you mention in your post, so how am i supposed to get the segments in direct stream copy without the 3d letters on them ? Yes, that's right... need to recode the whole segment, while in xvid this isnt necessary, i don't see how is this better...

Ok just tell me what you would do in my place... you would recode the entire segments just to remove the 3d letters ? Or search trough the whole episode for all the positions of the 3d leters beginnings and endings to indicate as scene cuts ? The problem is that like this not only is a lot slower than in direct stream copy, but also later i will use filters on these segments to move the damned scrolling tickler or at least to cover the letters with a blue color, i don't do this right now since i can't keep the 3,4gb original captured files right now, and i don't have time to be tweaking such filters at the moment, that's why i would like to avoid recoding...

[Dark Shikari]

Yes, that's right... need to recode the whole segment, while in xvid this isnt necessary, i don't see how is this better...Xvid isn't guaranteed to get all the scenecuts for you either.

x264's scenecut algorithm is optimized for compression, not getting every scenecut no matter what. That's one of the sacrifices you have to make when switching to an encoder that doesn't suck.

CRF ? Hum... the problem is knowing which is the best value for me, and usually everytime i tried other methods like this in 1pass encoding, when i tryed to make them have the same filesize as the one in ABR, it would ALWAYS have worse quality...CRF is vastly better than ABR and practically identical to 2-pass at the same bitrate.

Also, 2-pass does not take significantly more time than 1-pass; the first pass can be run with very fast settings (and is done so in most sanely designed frontends).

[LoRd_MuldeR]

CRF ? Hum... the problem is knowing which is the best value for me

And then you use ABR? :scared:

Specifying a certain target bitrate for an ABR encodes gives you absolutely no way to predict how the result will look.
That's because different sources have completely different bitrate requirements to look acceptable.
Of course you can specify and absurdly high bitrate, so all sources will look okay. But that's a huge wast of diskspace.

The great thing about CRF is that a specific CRF value will give similar quality, no matter what source you throw on it.
So once you found the CRF value that satisfies your eyes, you can use that value for all future encodes.
Also CRF will decide the required bitrate for you, so you don't need to blindly guess some bitarte for your ABR encode...

[cesm23]

And then you use ABR?

Specifying a certain target bitrate for an ABR encodes gives you absolutely no way to predict how the result will look.
That's because different sources have completely different bitrate requirements to look acceptable.
Of course you can specify and absurdly high bitrate, so all sources will look okay. But that's a huge wast of diskspace.

The great thing about CRF is that a specific CRF value will give similar quality, no matter what source you throw on it.
So once you found the CRF value that satisfies your eyes, you can use that value for all future encodes.
Also CRF will decide the required bitrate for you, so you don't need to blindly guess some bitarte for your ABR encode...


Hum... i see thanks a lot for the information... (i only read your reply after writing the part below) You see the big problem in this, is that i were unable to find this amount of details in a method without having to spend hours searching on google or in the forum, because there is so much information that even using search it still takes a lot of time.

But still try replying too to my doubts about the x264 encoder, the interlacer and the re-coding part i am so "afraid" off :)

Xvid isn't guaranteed to get all the scenecuts for you either.

Well... as i told you a few times in this post... that was quite rare and believe me i already did this type of editing in hundreds of recordings in xvid and it was REALLY very rare to not find a keyframe where i needed it... And in the very few occasions that happened, i just used smart rendering in vdub

x264's scenecut algorithm is optimized for compression, not getting every scenecut no matter what. That's one of the sacrifices you have to make when switching to an encoder that doesn't suck.

Jeez, i have no choice then, the only way is to try to guess which are the best encoding settings for x264 and encode this already as best as possible. But now this raises a few problems i hope you or someone else can help me on (please forgive me for asking so much stuff..). One of them is already that part of we not being allowed to ask which "is the best" here in the forums, like this how do i know if i am doing wrong or not, for example i always tought ABR was a very good one so you see this thing of not being able to ask which is the best is very restricting. Anyway i will still say the problems now that i have to switch to non vfw encoding :

- which generally is the most recommended deinterlacer ? (i presume that when encoding in x264 we can only use avisynth filter deinterlacers, right?) and that usually works better with the default settings ? So far i found out the adaptive interpolation method in pegasys tmpeg xpresss to be the one that best deinterlaces my videos, since most of the other avisynth filers i tried always still missed some parts of the video in the deinterlacing process, and the problem is that even if i tweak the settings it would be always different for each video this is why i would need one that works quite well with default settings.

- in case i am unable to find a suited deinterlacer for my captures, i will be forced to use tmpeg xpress's x264 encoder (because this sofware has the deinterlacer that best worked for me so far, interpolation adaptive method), which is named Main concept mpeg-4 encoder, i hope i am not using a encoder that "sucks" when compared to megui ? lool

CRF is vastly better than ABR and practically identical to 2-pass at the same bitrate.

Wow thanks to you and lord mulder for this them :) I just tried it now, and i found out that to give the same filesize than the abr at 1500 i was using, was using a rate of 15. I compared the image between the two files, even tough i haven't noticed much difference, at least it's not worse... LOL I will be using this mode from now on them.

By the way, and this is very important : when encoding video, and then later considering i have to encode it again (probably using CRF again) using some filters, is this a good method to use right now to avoid much quality loss when recoding ?

Also, 2-pass does not take significantly more time than 1-pass; the first pass can be run with very fast settings (and is done so in most sanely designed frontends).

Well, using xvid it used to take, i already did some 2 pass encoding in xvid and the first pass usually took as much time as the 2nd one... but of course in x264 it must be a lot better optimized :) But probably 2 pass encodings are harder to learn to which are the best settings so i guess i will stick with CRF for now, besides remember the slow cpu that i have, it's really not much suited for 2 pass encodings...

[Dark Shikari]

251, 467, 828, 1286, 1566x264 got all these except 251, which was definitely not a scenecut (a whole third of the image was unchanged from the previous frame).

You cannot rely on scenecut detection to put IDR frames at fade/zoom/similar transitions.

[cesm23]

x264 got all these except 251

??? How did you checked that exactly ? The only way i have to check which keyframes the x264 has is by splitting the mkv in pieces of 1s in mkv merge, unless it's wrong doing like this ?

When i split this mkv like that, it gave me 12 pieces, and none of them started with the 3d animated letters that start at those keyframes, what i did wrong then?

[Dark Shikari]

??? How did you checked that exactly ? The only way i have to check which keyframes the x264 has is by splitting the mkv in pieces of 1s in mkv merge, unless it's wrong doing like this?I used Elecard Streameye (free trial (http://www.elecard.com/products/products-pc/professional/streameye/)), a nice little stream analyzer, but given you only need to look at frametypes, there's far simpler (and cheaper/free) stuff around.

[cesm23]

What the ??? I used the same program as you and it says it's unsuported file format ?? impossible... i am really tired of so much problems about this, maybe it's best i move on to crf x264 encoding then, i don't know anymore what to do... i will try other programs to check keyframes then.

[LoRd_MuldeR]

Hum... i see thanks a lot for the information... (i only read your reply after writing the part below) You see the big problem in this, is that i were unable to find this amount of details in a method without having to spend hours searching on google or in the forum, because there is so much information that even using search it still takes a lot of time.

There are various x264 guides that will help you to understand the encoding methods. For example this one:
http://avidemux.org/admWiki/index.php?title=H264#General

[cesm23]

ok thanks a lot :)

The other problem now is finding a deinterlaced that works as good as the one in tmpeg or even better if possible, because don't tell me it doesn't exist a single deinterlacer that doesn't require user tweaking for every single video we have to deinterlace, this is absolutly out of question... and unfortunately that's what happened with most avisynth deinterlacers, everytime they worked well in one video, they didn't in another. At least the one in tmpeg didn't caused me such problem, altough i would prefer using other x264 encoders because the internal one of tmpeg (mainconcept mpeg4 encoder) doesn't seem to support CRF.

[LoRd_MuldeR]

The other problem now is finding a deinterlaced that works as good as the one in tmpeg or even better if possible

Try YADIF (http://avisynth.org.ru/yadif/yadif.html) or TDeint (http://avisynth.org.ru/docs/english/externalfilters/tdeint.htm). You can even use Avisynth's internal Bob() for a very simply method that just works ;)

What the ??? I used the same program as you and it says it's unsuported file format ?? impossible... i am really tired of so much problems about this, maybe it's best i move on to crf x264 encoding then, i don't know anymore what to do... i will try other programs to check keyframes then.

Probably Streameye requires an elementary (raw) H.264 stream, instead of an AVI file. Hence you must demux the H.264 stream from the AVI container first.

(Assuming you are still encoding in VirtualDub and hence got an AVI file)

[Dark Shikari]

What the ??? I used the same program as you and it says it's unsuported file format ?? impossible... i am really tired of so much problems about this, maybe it's best i move on to crf x264 encoding then, i don't know anymore what to do... i will try other programs to check keyframes then.It doesn't read MKV. Yes, you have to extract the .h264 file. This isn't hard.ok thanks a lot :)

The other problem now is finding a deinterlaced that works as good as the one in tmpeg or even better if possible, because don't tell me it doesn't exist a single deinterlacer that doesn't require user tweaking for every single video we have to deinterlace, this is absolutly out of question... and unfortunately that's what happened with most avisynth deinterlacers, everytime they worked well in one video, they didn't in another. At least the one in tmpeg didn't caused me such problem, altough i would prefer using other x264 encoders because the internal one of tmpeg (mainconcept mpeg4 encoder) doesn't seem to support CRF.Oh come on, there are dozens of good deinterlacers, and I have never even heard of one that requires "user tweaking."

[cesm23]

It doesn't read MKV. Yes, you have to extract the .h264 file.
Oh that... ok.

Oh come on, there are dozens of good deinterlacers, and I have never even heard of one that requires "user tweaking."

Really ?? That's not what happened when i tried them, the few that i tried all had that problem i complain, they worked in a few videos and in others i had to try several values in the settings to make the deinterlacing being correctly done on each video :

TomsMoComp.dll
TDeint.dll
decomb523

Right now i am trying Yadif, let's see if it works, but i am not surprised if i find a video that already requires different settings.

Anyway i see that the only way is to me try ALL that do exists and only use one that requires no tweaking to work effectively, because all the three above did not worked in every video, they deinterlaced most of the video ok, but in certain parts it was not, and only worked well in those parts after tweaking the settings, i suppose this isn't normal right ?

Try YADIF or TDeint. You can even use Avisynth's internal Bob() for a very simply method that just works

Yes yadif it's possible that might work for me, but even like that i get several doubts since they say in forums that it's better use method 1 and that might lose quality or get artifacets with method 0 (it's because things like this that i get so confused, how do i know were is reliable information, no wonder hy i have to make all these questions), i hope it's not necessary to be changing settings too for every single video (and by the way i need to keep the videos at 25fps, of course i don't want them at 50fps). TDeint is one of those i already tried and without user settings it does NOT deinterlace some parts of the videos i tried.

[LoRd_MuldeR]

The only difference between "mode=1" and "mode=0" of Yadif is that the former will keep the framerate (and consequently discard half of the temporal information), while the latter will double the framerate (bob mode). Of course "mode=1" will give more smooth motion (and hence look more appealing), but also the bitrate requirement of your encode will increase significantly...

[cesm23]

ok i see indeed so far it was the best deinterlacer i found for avisynth, let's hope it deinterlaces everything correctly, because i am tired of using deinterlace filters that are unable to deinterlace video completly.

Ok then, i think for now i will start using megui, which i think it's the most recomended x264 encoder, right ? And using the yadif deinterlacer, along with CRF method encoding, now the real problem is finding out the best settings for x264 besides the method, since even with the guides, well it's not a easy task finding out the best combination of quality and size for my encodings.

The only doubt for now is still the reenconding part, i hear SO frequently everywhere that there is quality loss when recoding videos, that's why i asked that about the CRF mode as well, since i don't seen anything specific about this in the guides, i guess the best way is try for myself (and waste hours and hours on this, lool).

[LoRd_MuldeR]

Ok then, i think for now i will start using megui, which i think it's the most recomended x264 encoder, right ?

Use the front-end you like best. I recommend you give Avidemux (http://forum.doom9.org/showthread.php?t=126164) a try as well. There are many others (RipBot264, StaxRip, Handbrake, etc).

The only doubt for now is still the reenconding part, i hear SO frequently everywhere that there is quality loss when recoding videos, that's why i asked that about the CRF mode as well, since i don't seen anything specific about this in the guides, i guess the best way is try for myself (and waste hours and hours on this, lool).

Re-encoding inherently hurts quality, unless you encode to some lossless format. The latter would produce a file MUCH bigger than the source, of course. So if you are re-encoding to a lossy format (such as H.264) the best thing you can achieve is minimizing the additional loss down to the point where you can hardly spot it (aka "transparent" quality). And CRF mode definitely is your best tool for achieving this!

[cesm23]

Well after many tests i just did now i think that it's still better to me use the vfw version of the x264 codec. Also i found out that when i converted the abr 1500kbps encoding to crf at 12.0 i finally have avoided loss of quality in the recoding ! this is great, this way i will only recode these videos in crf at the end when doing the final filtering.

I hope there isn't any big losses in quality just because using the vfw version of x264 instead of the non-vfw right ?

[LoRd_MuldeR]

Well after many tests i just did now i think that it's still better to me use the vfw version of the x264 codec.

And what makes you think that? :confused:

Also i found out that when i converted the abr 1500kbps encoding to crf at 12.0 i finally have avoided loss of quality in the recoding !

CRF 12.0 sounds a bit overkill. Usually people get "transparent" encodes at CRF 16.0 or even higher.

(Also: With your initial ABR encode you also wasted a lot of quality. That quality won't come back, no matter what you do afterwards)

[cesm23]

And what makes you think that? :confused:

Just nevr mind that part i already know now how to use megui the way i need, finally.

CRF 12.0 sounds a bit overkill. Usually people get "transparent" encodes at CRF 16.0 or even higher.

Well you see, with the inicial encoding at ABR 1500kbps... i reencoded it in CRF and i did a very detailed comparation and i could perfectly see that ONLY at 12.0 it did not gain extra blocks... at 12.5 or above i started to notice some blocks appearing... I guess this is different in each video, or then this is caused by the inicial encoding at abr 1500kbps, probably that's why it's necessary a setting of 12 in crf with these encodings.

(Also: With your initial ABR encode you also wasted a lot of quality. That quality won't come back, no matter what you do afterwards)
By the way when you say that, you took in account that the original capture was in mpeg2 720x576 10000kbps bitrate?

But yes i am aware of that, altough with a cfr setting of 13 it was already getting WORSE than the original encoding at abr 1500, probably because of the quality that was lost when encoded at abr 1500.

[LoRd_MuldeR]

By the way when you say that, you took in account that the original capture was in mpeg2 720x576 10000kbps bitrate?

Depending on the content of your 10 mbps MPEG-2 source, 1500 kbps may be sufficient for x264 to retain a decent quality or it may be MUCH too low.

But given that you were using Single-Pass ABR, which uses the available bitrate far less efficient than a proper 2-Pass encode, I assume there was a significant degradation.

Hence you would be much better off by doing a CRF 18-22 encode directly from the original MPEG-2 source, instead of re-encoding the shabby ABR encode once again...

[cesm23]

Yes that's what i am going to test, i just tested using the default value of cfr (26) and jeez the quality is absolutly terrible :) Even tough i am using that mpeg2 as the source this time, altough i am using a avisynth script like this :

LoadPlugin("h:\Filtros\DgDecode\DGDecode.dll")
Load_Stdcall_plugin("H:\Filtros\yadif\yadif.dll")
LoadPlugin("H:\Filtros\NicAudio_r202\NicAudio.dll")
audio=NicMPG123Source("D:\Videos por encodar\nico\56658_8_0509_1701-1 Tc0 L2 2ch 48 384 DELAY 106ms.mp2")
video=mpeg2source("d:\videos por encodar\nico\56658_8_0509_1701-1.d2v")
audiodub(video,audio)
yadif(mode=0,order=0)

LanczosResize(384,288)

Trim(8851, 57007)

I hope this is correct ? lool

(by the way when i said it was terrible at 26, that's when i compared the same encoding using abr at 1000kbps this time using this same script as well)

[Chengbin]

Why are you resizing it to 384x288?

If you are, 1500Kbps is more than enough, overkill actually.

And ALWAYS encode from the best source. A 1500Kbps ABR encode as a source is quite bad.