Weird prob with AVIs containing x264 video

[JimmyBarnes]

When I open one of these in VirtualDubMod 1.5.10.2 it gives a warning: "Couldn't locate decompressor for format 'H264' (unknown). VirtualDub requires a Video for Windows (VFW) compatible codec to decompress video. DirectShow codecs, such as those used by Windows Media Player, are not suitable. Only 'Direct stream copy' is available for this video."

x264vfw is in fact installed and used quite happily by TMPGEnc Video Mastering Works 6.

So AVIs can be cut etc. and saved using direct stream copy using VDubMod, only prob being video is not displayed.

VirtualDub FilterMod 38919 can open and display AVIs with x264 video, but is unable to use direct stream muxing: "The source video stream uses a compression algorithm which is not compatible with AVI files. Direct stream copy cannot be used with this video stream."

Truly weird.

I recently migrated from WinXP to Win10. In WinXP, VDubMod with x264vfw worked without probs. It could display/decode videos with x264/h264 video and cut them etc. using direct stream copy or full processing mode etc.

Any suggestions? (no I do not want to convert these AVIs to MKV)

[raffriff42]

VirtualDub FilterMod 38919 can open and display AVIs with x264 video, but is unable to use direct stream muxing: "The source video stream uses a compression algorithm which is not compatible with AVI files. Direct stream copy cannot be used with this video stream."That is actually true: although h.264 has been shoehorned into AVI before, it is not recommended (https://forum.doom9.org/showthread.php?t=143613). VdubFM is saving you from yourself ;)

Any suggestions? (no I do not want to convert these AVIs to MKV)Given the preconditions, Nope, none at all.

[JimmyBarnes]

That is actually true: although h.264 has been shoehorned into AVI before, it is not recommended (https://forum.doom9.org/showthread.php?t=143613). VdubFM is saving you from yourself ;)

So its an administrative constraint?

In the past 7 years I have backed-up hundreds of blu-rays to AVI/x264. Those backups invariably play without problem.

[JimmyBarnes]

That is actually true: although h.264 has been shoehorned into AVI before, it is not recommended (https://forum.doom9.org/showthread.php?t=143613). VdubFM is saving you from yourself ;)

Also VDubFM offers x264 as a compression type.

x264 10-bit - H.264/MPEG-4 AVC codec
x264 8-bit - H.264/MPEG-4 AVC codec

[raffriff42]

OK I was wrong. I found an x264 AVI on my end that can not be stream-copied in VirtualDub FM with the native decompressor, but can when forced into using the x264vfw decompressor (via File, Open, files of type, AVIFile input driver).

[JimmyBarnes]

OK I was wrong. I found an x264 AVI on my end that can not be stream-copied in VirtualDub FM with the native decompressor, but can when forced into using the x264vfw decompressor (via File, Open, files of type, AVIFile input driver).

File, Open, files of type, AVIFile input driver doesn't see AVI files but it does see AVS, so I tried this with an avs containing the single line:

DirectShowSource("x264 AC3 Hugo 1360x768.avi")
where x264 AC3 Hugo 1360x768.avi is x264 video, AC3 audio 27,740,160 bytes.

The AVS opens and displays in VDFM OK using File, Open, files of type, AVIFile input driver and will even Direct Stream Copy without error, but the result is an AVI of 2,278,020,244 bytes, clearly some form of uncompressed video.

If you got a Direct Stream Copy AVI result almost identical in size to your input AVI, please tell me what you did.

[raffriff42]

AVIFile input driver does not see AVI files? Hmmm yeah. What I did was copy & paste a path without even trying to browse (I had to search for a suitable AVI)

An Avisynth file appears as a raw, uncompressed stream, so copying it will result in a huge file.

EDIT and the copied video stream has exactly the same size and other properties. Audio copying failed, but that is another story...

[JimmyBarnes]

AVIFile input driver does not see AVI files? Hmmm yeah. What I did was copy & paste a path without even trying to browse (I had to search for a suitable AVI)

In VDFM, "AVIFile input driver (compat.)" lists the files it can open as (*.avs, *.vdr, *.vpy) - simply doesn't list any AVIs.

HOW did you "copy & paste a path without even trying to browse". Into WHAT did you paste this path?

[raffriff42]

I ran a search for *.avi, picked one at random, shift-clicked it (Win 8/10), "copy as path", clicked in the box directly above Files of type, right-click, Paste.

[hello_hello]

For the standard VirutalDub, "Audio/Video Interleaved" is included in the drop down list when opening files. "AVIFile input driver" doesn't include AVIs. Try the standard version if you're using the modded version, but it probably works the same way as the standard version with the ffmpeg input driver plugin installed and you need to select "Audio/Video Interleaved". For the standard VD, without the ffmpeg input driver plugin, it should open AVIs in the usual way. If the plugin is present, it'll probably default to opening the files so you won't be able to use directstream copy unless you select "Audio/Video Interleaved". Even if VD complains about not being able to find a codec for decompression, you should still be able to use direct stream copy if you open them that way.

https://www.videohelp.com/software/Virtualdub
https://sourceforge.net/projects/virtualdubffmpeginputplugin/

[JimmyBarnes]

I ran a search for *.avi, picked one at random, shift-clicked it (Win 8/10), "copy as path", clicked in the box directly above Files of type, right-click, Paste.

"Copy as path" is not available to me, how did you get it?

[JimmyBarnes]

For the standard VirutalDub, "Audio/Video Interleaved" is included in the drop down list when opening files. "AVIFile input driver" doesn't include AVIs. Try the standard version if you're using the modded version, but it probably works the same way as the standard version with the ffmpeg input driver plugin installed and you need to select "Audio/Video Interleaved". For the standard VD, without the ffmpeg input driver plugin, it should open AVIs in the usual way. If the plugin is present, it'll probably default to opening the files so you won't be able to use directstream copy unless you select "Audio/Video Interleaved". Even if VD complains about not being able to find a codec for decompression, you should still be able to use direct stream copy if you open them that way.

https://www.videohelp.com/software/Virtualdub
https://sourceforge.net/projects/virtualdubffmpeginputplugin/

Thanks for your reply, but see the OP.

I would like a SINGLE app which can both open AVIs and view them (so I can select regions to cut etc.) AND do Direct Stream Copy. At the moment I have to use VDM and VDFM.

[mariush]

virtualdub can do that but can only cut at specific frames (keyframes), not at any point.
Sometimes there's only a keyframe every 10-15s (250-300 frames is common default in encoders), so you wouldn't be able to cut such series of frames in the middle.
You can cut at exact frames with some supported codecs if you use "smart recompress" mode (or whatever it's called) instead of direct stream copy.
virtualdub would copy segments outside cut points directly (like direct stream copy) and decode segments which have cut point inside and recompress the portion you still want in video.

but h264 in avi is already such a hack i wouldn't be surprised if "smart recompress" won't work.
Really, stop using avi and use mp4 or mkv

[JimmyBarnes]

virtualdub can do that but can only cut at specific frames (keyframes), not at any point.

I mainly do downstream cuts which VDM can do at any given frame, unlike in an MKV where even a downstream cut occurs at a KF.

As I say I have done hundreds of AVI/x264 rips without problem, that's the bottom line...

I could convert AVI to MKV at any time, but not the reverse, another thing I dislike about MKV.

[raffriff42]

"Copy as path" is not available to me, how did you get it?I did say "shift-click", which I admit could have been clearer. Should have been "shift-RIGHT-click" anyway.
For a better explanation, Google Windows X copy as path. (where X = your Windows version)

[mariush]

I mainly do downstream cuts which VDM can do at any given frame, unlike in an MKV where even a downstream cut occurs at a KF.

As I say I have done hundreds of AVI/x264 rips without problem, that's the bottom line...

I could convert AVI to MKV at any time, but not the reverse, another thing I dislike about MKV.

MKV is just a container, just like AVI.

It packs one or several video streams, one or several audio streams, one or several subtitles and their related data (fonts, various resources) and chapter information into a single file.

AVI is worse, it supports only 2 audio streams (good luck storing original audio and multiple commentary tracks in one avi file), it has problems with large sizes (some players can't play AVI files larger than 8 GB or crap like that), there's index at the end so if you don't have the complete AVI file you can have problems playing the video... it's crap container.

The limitation of cutting just at a keyframe with MKV is probably just some lazyness from the developer's part - it's easier that way, it's like i said direct stream copy and just editing the "index" of the MKV file (a collection of seek points which tell video players to jump to specific keyframes in the video).

MKVToolnix (https://mkvtoolnix.download/) is a popular tool to mux streams in matroska containers or to extract segments from matroska files and it only cuts at keyframes. So it's quite a high possibility the software you're using relies on this tool in the background to edit MKV files.

x264 in AVI is done as a HACK. Even XVID when it was popular was implemented in AVI as a hack, because AVI as a movie container has some limitations and couldn't normally support all the "features" of modern video codecs like XVID or h264.

You would most likely get better quality or more quality per bitrate if you'd encode some video to a raw h264 stream or to MKV or MP4 (x264vfw codec even let's you output the encoded data to a separate file in a format it supports and return some fake video back to Virtualdub) , and then mux that video-only file with the audio compressed separately to MP3 or AAC or Opus or FLAC (or whatever you prefer) and mux both into MKV or MP4.

You can't do MP4 to AVI or MKV to AVI most likely for the reason that the bitstream of the video would have to be specially crafted to fit inside AVI, and there's no guarantee the stream that's in MP4 or MKV was produced like that originally. And no developer cares to somehow find a hack or some way to put that h264 data from mkv or mp4 to AVI because as you've been told already, it wasn't meant to be used with AVI and AVI should just die.

[JimmyBarnes]

I did say "shift-click", which I admit could have been clearer. Should have been "shift-RIGHT-click" anyway.
For a better explanation, Google Windows X copy as path. (where X = your Windows version)

Hmm.. interesting, I got the above to work as you say but the only response in VDFM was "Couldn't locate decompressor for format 'H264' (unknown)" and the file doesn't load at all. :(

[JimmyBarnes]

MKV is just a container, just like AVI.

I understand that dude, it's a wrapper..

The advantages of MKV, I don't use:
* Never use 2 audio streams
* Don't use files sizes above 8 GB (mostly under 4 GB)
* Never use chapters

Even embedded subtitles which is something I find appealing haven't worked well last time I looked. There is a marked "judder" when turning subtitles off and on which does not occur (in MKV or AVI) with external subtitles.

Being able to cut at a given downstream frame is a big deal to me, so sorry MKV...

I had this discussion with a leading MKV proponent (moderator) before. In the end he admitted there was no absolute reason I should leave AVI given what I want and how AVI is performing. :eek:

[sneaker_ger]

but see the OP.
What is the supposed to see in the start post about vanilla VirtualDub not being an option? :confused:

Is the decoder component of x264vfw deactivated? Which version/from where? Tried to install it again? Did you try ffdshow vfw?

I would like a SINGLE app which can both open AVIs and view them (so I can select regions to cut etc.) AND do Direct Stream Copy. At the moment I have to use VDM and VDFM.
H.264-in-AVI is a bit special but softwares like SolveigMM Video Splitter, VideoReDo and AviDemux are supposed to handle cutting H.264 correctly. (Including in MKV container.)

[JimmyBarnes]

Is the decoder component of x264vfw deactivated? Which version/from where? Tried to install it again? Did you try ffdshow vfw?

Yes.
x264vfw_full_43_2694bm_43159_fix
Not yet but TMPGEnc Video Mastering Works 6 is quite happy with x264vfw as installed.
No.

H.264-in-AVI is a bit special but softwares like SolveigMM Video Splitter, VideoReDo and AviDemux are supposed to handle cutting H.264 correctly. (Including in MKV container.)

I have tried some in the past when I contemplated switching from AVI to MKV. Ones I tried then which claimed to be able to cut an any downstream (or maybe given) frame did so by recoding a small section around the cut frame.

[sneaker_ger]

Ones I tried then which claimed to be able to cut an any downstream (or maybe given) frame did so by recoding a small section around the cut frame.
If the inter-frames compression around that frame requires it that's a given. It's not possible otherwise.

[JimmyBarnes]

If the inter-frames compression around that frame requires it that's a given. It's not possible otherwise.

Mainly referring to downstream cutting i.e. at the very end of a clip, where it should be no big deal to truncate frames from any given frame through to the end of the clip.

[JimmyBarnes]

OK, everything is always easier in hindsight... :rolleyes:

Being a longtime WinXP x86 user who'd only recently upgraded to Win10 x64, I assumed that when 32- and 64-bit versions of a program were on offer, the 64-bit one should be used.

So when I originally installed x264vfw and it offered both 32- and 64-bit versions, I chose only the 64-bit one (I keep a log).

As sneaker_ger suggested, I reinstalled x264vfw, but this time both 32- and 64-bit versions.

Now VDubMod 1.5.10.2 opens AVI with x264 without problems, the video stream can be viewed, cut points nominated and the result saved using Direct Stream Copy - just as it used to do under WinXP. :D

Clearly VDM is a 32-bit prog which can't use 64-bit codecs.

Curiously VDFM still won't do Direct Stream Copy, but I don't need it now...

Thanks to everyone for their contributions.

[JimmyBarnes]

Is the decoder component of x264vfw deactivated?

BTW What does deactivating the decoder actually do?

The default setting is enabled and normally I have always used it like that.

[Midzuki]

.............

AVI is worse, it supports only 2 audio streams (good luck storing original audio and multiple commentary tracks in one avi file)

Speaking in the name of Alex Noe :devil:

AVI can only handle 1 or 2 audio streams

Totally wrong. A few people who think to write container comparisons still believe this, or even write such disinformation in order to make up advantages of OGM that do not exist, although even the original AVI file specification supports 256 streams (no matter how many of them are audio or video), and although AVI-Mux GUI has been able to create AVI files with multiple audio streams for years

and AVI should just die

After Ogg gets dead and incinerated, AVI may be killed soon afterwards :)

[nevcairiel]

The real reason AVI should not be used for an advanced codec like H.264 is that it just doesn't support B-Frames properly (as mariush said, even XVID already had this problem and they used even worse hacks to encode "packed B-Frames" in AVI), you don't get correct timestamps and decoders need to implement all sorts of hacks to somehow "guess" timestamps for H.264 in AVI (which sometimes works, but sometimes doesn't, resulting sync problems). The real question is why would you prefer AVI when you can use something much better suited to store H.264?

[sneaker_ger]

Mainly referring to downstream cutting i.e. at the very end of a clip, where it should be no big deal to truncate frames from any given frame through to the end of the clip.
You don't think it be like dis but it do. You cannot always arbitrarily cut H.264 even when setting end points on account of the re-ordering.

[shekh]

The real reason AVI should not be used for an advanced codec like H.264 is that it just doesn't support B-Frames properly (as mariush said, even XVID already had this problem and they used even worse hacks to encode "packed B-Frames" in AVI), you don't get correct timestamps and decoders need to implement all sorts of hacks to somehow "guess" timestamps for H.264 in AVI (which sometimes works, but sometimes doesn't, resulting sync problems). The real question is why would you prefer AVI when you can use something much better suited to store H.264?

here https://en.wikipedia.org/wiki/Comparison_of_video_container_formats#Video_formats_supported
there is a footnote 41 "B-frames in an AVI file are a problem only for the ancient Video-for-Windows API, not for the AVI container itself."

I've no idea who wrote it, but do you think this article is wrong? Can you name usage case when h264 in avi is wrong, given that all the following is met:
* CFR
* encoding and decoding is done with something that knows what to do, e.g. ffmpeg.exe

asking because of curiosity.

[JimmyBarnes]

You don't think it be like dis but it do. You cannot always arbitrarily cut H.264 even when setting end points on account of the re-ordering.

I must be extraordinarily lucky then, hundreds of blu-ray to AVi/x264 rips scrutinised carefully without visible problems. I should clearly go and buy a Lotto ticket... :sly:

[sneaker_ger]

Ignorance is a bliss

[JimmyBarnes]

Ignorance is a bliss

You don't like empirical evidence, dogma rules..

[sneaker_ger]

If you cut through references you end up with stray/missing slices. That's just the way it is. So if softwares specifically designed to edit H.264 re-encode on such cut points you can assume they do it for a good reason. If VirtualDub that was designed around VfW does not do that it should raise your eyebrows. Playback behavior on such points can be erratic.

[JimmyBarnes]

If you cut through references you end up with stray/missing slices. That's just the way it is. So if softwares specifically designed to edit H.264 re-encode on such cut points you can assume they do it for a good reason. If VirtualDub that was designed around VfW does not do that it should raise your eyebrows. Playback behavior on such points can be erratic.

But that's the whole point, such problems are NOT observed... If they were, I would have long sought something better.

Look, I got the answer to my problem, its clear this has become a a discussion on the merits or otherwise of MKV / AVI.

You are convinced your POV is correct just as I am mine, there is no point continuing such a discussion.

[burfadel]

AVI also doesn't support formats like AAC, and has a larger overhead than MKV. Cutting at keyframes is done because cutting anywhere else would effectively result in a corrupted stream. They got around this in the past by cutting at the keyframe then re-encoding to the cutpoint, then appending it to the original stream. It's messy and not really a direct steam copy. Even x264 through the VFW interface was a hack and not supported by the x264 development team. Support came through unofficial hack. Only reason to use AVI in 2012, let alone 2017, is for old stand alone players that don't support MKV or MP4, but even these would support at least MP4 if they support h264.

Virtualdubmod is ancient from a software perspective, and VFW is deprecated and will never be updated apart from it being removed entirely; I thought it already had. I would consider updating your whole lot of tools. Why the need to make all the cuts in the video? Reasons for your specific needs would be helpful. Why the need for encoding then cutting through direct stream copy? Why not cut when doing the initial encode?

[sneaker_ger]

AVI also doesn't support formats like AAC
http://www.alexander-noe.com/video/amg/en_myths.html
http://www.alexander-noe.com/video/documentation/containers.pdf

[Andouille]

<pengvado> <Dark_Shikari> if a fansubber or DVD rip group uploaded an H.264-in-AVI file they'd get laughed off the internet
<Dark_Shikari> Probably
<pengvado> but asp-in-avi wouldn't be laughed at
<Dark_Shikari> of course not, ASP in AVI is normal unless you want softsubs
<pengvado> and asp-in-avi requires exactly the same ugly hacks
<Dark_Shikari> Probably because its been that way so long that everyone is accustomed to it
<pengvado> which just goes to show that our campaign to use a new codec as an excuse to tell people to upgrade their container is working

From http://www.x264.nl/developers/Dark_Shikari/loren.html

[burfadel]

Maybe so, but those quotes are from several years ago.

[hello_hello]

But that's the whole point, such problems are NOT observed... If they were, I would have long sought something better.

Look, I got the answer to my problem, its clear this has become a a discussion on the merits or otherwise of MKV / AVI.

You are convinced your POV is correct just as I am mine, there is no point continuing such a discussion.

Unless someone tests your claims.

I ran a few little encodes with the vfw version of x264 and VirtualDub. They were each encoded with the default x264 settings. I only changed the output options or checked the zero latency option.

My test encodes had 418 frames. When I used the "write to a file" option, the first keyframe was at frame zero, and the next at frame 98. I cut the first 50 frames and saved a new AVI using direct stream copy. I did the same for the other test encodes.

For the encode written directly to a file, nothing was cut and the new AVI still had 418 frames.
The result was exactly the same for the encode with the VirtualDub hack enabled and using the vfw output method, whatever that does. The first 50 frames couldn't be cut.

For the vfw encode without the VirtualDub hack enabled, the first 48 frames weren't encoded. It looks like they were replaced with null frames. The next keyframe was now at frame 147, so I moved my cut point up to frame 110. After saving with direct stream copy the new AVI should have had 380 frames. It had 369, so obviously VirtualDub simply moved the cut point up to the next available keyframe.

The zero latency option seems to just disable B frames, so the bitrate increases quite a bit, but still didn't let me cut at frame 50.

I'd be interested to learn the incantation you're using when encoding that permits cutting anywhere.

I've nothing against the AVI container but I use MKV myself because I don't think a single hardware player in our house supports h264 in an AVI, but they're happy to play it in an MKV or MP4. That's reason enough for me even if AVI had magic properties that allowed cutting on non-keyframes without ill effects, but the keyframe cutting problem is mostly a minor inconvenience and easy enough to work around.

[raffriff42]

Good testing there hello_hello. By the way, according to the developers, x264 optimized for zero latency reduces all the following sources of latency:Diary Of An x264 Developer 01/13/2010
x264: the best low-latency video streaming platform in the world (http://wayback.archive.org/web/20150507012544/http://x264dev.multimedia.cx/archives/249)

The total latency of x264, including encoder/decoder-side buffering, is:
B-frame latency (in frames) + Threading latency (in frames) + RC-lookahead (in frames) + Sync-lookahead (in frames) + VBV buffer size (in seconds) + Time to encode one frame (in milliseconds)I don't know if the "Zero latency" checkbox does all that or not. None of these optimizations help a file-based (not live-streamed) video anyway - they only increase file size.

[hello_hello]

raffriff42,
It seems to be the same zero latency, at least to a certain extent. I read the post you linked to and while I don't understand it all fully yet, some of the relevant x264 settings obviously changed when zero latency was enabled. B Frames disabled, Slices enabled, Threads reduced, Lookahead threads increased. Nothing that'd make it possible to cut on any frame.

A small SD test encode using the default settings. x264 set High Profile, Level 3 each time. An old 4 core CPU with no hyper-threading used for encoding.

Without the zero latency checked:

cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

With the zero latency option checked:

cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=4 / lookahead_threads=4 / sliced_threads=1 / slices=4 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc=crf / mbtree=0 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

[sneaker_ger]

B Frames disabled, Slices enabled, Threads reduced, Lookahead threads increased. Nothing that'd make it possible to cut on any frame.
Without B frames you can cut at the end on arbitrary frames. OP was talking about something like that (post #22).
But of course solutions like SolveigMM Video Splitter should handle such cases without re-encoding as well.

[hello_hello]

Yes, I tried it without B-Frames and VirtualDub will cut an AVI on a non keyframe, as long as everything from the cut point to a later keyframe is removed (or from the cut point to the end).

A quick test with MKVToolNix indicates it sticks to splitting on keyframes but it's not really an editor and I guess the assumption is all the segments should be playable.

Still, cutting on any frame that way would require disabling B-frames or using the zero latency option, and the OP didn't indicate he was doing either.

[isidroco]

...I could convert AVI to MKV at any time, but not the reverse, another thing I dislike about MKV.

For MKV to AVI:
Usually I use MKVcleaver v0.7.0.2 (needs MKVtoolnix) to extract all streams from all MKVs (will make AVIs from h264 streams), and I use a batch for directStream muxing all videos and audios.
Batch: https://sourceforge.net/projects/virtualdub-batch-video-deshake/?source=directory
MKVcleaver: https://blogs.sapib.ca/apps/mkvcleaver/

[JimmyBarnes]

For MKV to AVI:
Usually I use MKVcleaver v0.7.0.2 (needs MKVtoolnix) to extract all streams from all MKVs (will make AVIs from h264 streams) [snip] MKVcleaver: https://blogs.sapib.ca/apps/mkvcleaver/

Thanks, I'll give it a go.. :D