Questions that need answering to further my encoding understanding

[pwnsweet]

Hi,

I'm fairly new to the video editing/encoding world. I've spent many full days (yes, days) reading, testing and researching and have come a long way in a short time but it has become apparent that I am lacking a basic understanding of important topics such as what a 'filter' is, what a 'frameserver' is and why we need things like Haali Media Splitter (or what Haali Media Splitter even does, for that matter).

Why I've never learnt these things yet is unknown to me but it seems I've somehow accidentally sidestepped these issue on my way to learning about x264 and encoding and now my lack of understanding is hindering my progress in learning the more advanced aspects of x264 encoding. Not only that, it's also limiting my ability to actively participate in discussion on this forum for fear that I might say something that makes me look stupid or, more importantly, say something that somebody with even less knowledge takes as fact but turns out to be wrong. So, I'd like to go back and fill in the gaps to better grasp these concepts. I'm going to do this by just throwing questions out there. Some of these questions are going to be outright stupid, and you'll probably wonder how I even managed to encode a single video without knowing the answer to them but I feel I need to ask them to clear up the confusion I have.

1. My first question is, what is Haali Media Splitter and what does it do? I noticed that every time I open a .m2ts Blu-ray file in AVS script creator (MeGUI), I get a FFv and Haali Media Splitter icon in my system tray. I've no idea why the Haali Media Splitter icon is there or what it is doing.

2. Following on from the previous question, why is the FFv icon there? Is it because ffdshow is decoding the .m2ts file? Is it because ffmpeg mt is decoding the .m2ts file? Is ffdshow the same thing as ffmpeg mt? What exactly is ffdshow? What exactly is ffmpeg mt?

3. I use a program called Win7DSFilterTweaker_4_1. According to its description, it allows you to change the preferred DirectShow decoding filters in Windows 7. I'm running Windows 7, but I've no idea what DirectShow is or what a decoding filter is. I assume a decoding filter is what renders a h.264/mpeg2/vc-1/etc video stream. Is this correct?

4. No question here. The purpose of this entry is just to state that I understand the concept of containers, muxing and demuxing and serves to save repliers to this post the trouble of explaining what these concepts are. There will be more of these kinds of entries as I go along so that repliers don't repeat things I already know.

5. Regarding Win7DSFilterTweaker_4_1, what settings should I be using for H.264, VC-1 and MPEG-2? I have a DXVA compatible videocard (Nvidia GTX460) and so they are currently set to ffdshow DXVA for H.264 and VC-1 respectively and ffdshow for MPEG-2. Is this 'correct'? As I understand it, ffdshow DXVA means the video decoding (ie, video playback) is computed on the videocard, so does that mean that the only benefit to setting it to ffdshow DXVA for H.264 and VC-1 is there will be less load on the CPU?

6. I understand frame rates, the concept of 2:3 pulldown and the concept of ivtc. I understand the concepts of CPU cores, SMT and threads. I understand the difference between YUV/YV12 and RGB colour.

7. When I used to encode a .m2ts file (with a H.264 video stream) in MeGUI, I used to get a message at the 'enqueue' stage (ie, when adding the file to the encode queue) that said the source file was not in YUV colour format and MeGUI would ask me if I wanted to add the 'ConverttoYV12() function to the Avisynth script. What I don't understand is, when I used MediaInfo on the file it said that the colour space was already YUV (which is, correct me if I'm wrong, the same as YV12). If this is true, then why was MeGUI not getting the source in YUV colour?

This used to happen before I knew about Win7DSFilterTweaker_4_1, and I resolved this by using Win7DSFilterTweaker_4_1 to change the H.264 option from 'Microsoft' to 'ffdshow DXVA'. I noticed that, after I did this, the FFv icon started showing up in the system tray whenever I opened the .m2ts file. I can only assume this is because ffdshow was now being using to decode the video file, but why did making this change solve the ConverttoYV12() problem?

8. Following on from the previous question, I recently came across a Blu-ray that uses a YUV MPEG-2 video stream. When I added it to the MeGUI queue, I received the 'ConverttoYV12()' message again. So, as before, I used Win7DSFilterTweaker_4_1 to change the MPEG-2 option from 'Microsoft' to 'ffdshow' (there was no ffdshow DXVA option for some reason - anyone know why?). This time, however, it didn't fix the problem. I still received the ConverttoYV12() message.

9. What is a 'frameserver', what are some examples of frameservers, do I need a frameserver and which is the best frameserver?

10. Besides the amount of load a decoder puts on the CPU, is there any image quality differences between different video decoders? If so, which H.264/AVC video decoder produces the highest quality image?

11. Which is the better ffdshow decoder to accomplish my goal of encoding H.264/VC-1/MPEG-2 .m2ts Blu-ray files to H.264 mp4 files, libavcodec or ffmpeg, and why is one better than the other?

12. Would the answer to Q10. change if CoreAVC's H.264/AVC decoder was also considered? From what I understand, CoreAVC allows the use of a CUDA enabled videocard (such as my GTX460) to decode H.264 streams. I know that as an H.264 encoder CUDA is hopeless, but how does it fare as a H.264 decoder?

13. Is Haali Media Splitter the best demux filter for my goal outlined in Q11, or should I be using something else? If so, why, and if not, why not?

14. What's the difference between DXVA and CUDA and which is better for H.264/AVC decoding?

15. Regarding 'filter chains', what would be the ideal filter chain to use if the goal is to maximize image quality, assuming I was not limited to free software only and had a CUDA enabled GPU at my disposal?

16. I've recently learnt about indexing (only that it exists). According to MeGUI, whenever I try to index a .m2ts file it automatically sets DGindex as the default indexer, however FFMSindex is an alternative indexer that is selectable. What is the difference between these two indexers, how I can know which one I should use to index my file and why?

*note*
As an aside, I noticed that I have the option to perform color correction when using DGindex to index my file and that the preview window in MeGUI AVS script creator is darker than the source. Not 0-255 vs 16-235 brightness levels amounts, but definitely darker. Color correction is required to make the colors in the preview window match that of the source but the darker image phenomenon is independent of whether color correction is on or off and I cannot change it. However, I noticed that if I use FFMSindex to index the same stream (but muxed as an .mkv instead of .m2ts) there is no color correction option, the colors are the same in the preview window as they are in the source and the darkness issue is non-existant.

17. What does it mean when somebody says "It's even possible to directly copy the a/v stream of wmv when it is wrapped as an internal decoder"?


I will add more questions as I think of them. Thanks go out in advance to all people who respond to this post.

[nhakobian]

I'm fairly new to the video editing/encoding world.


Welcome! There is no shame in asking questions and I think you'll find that many people will be more than willing to give you a hand in understanding many of the more complicated ideas. It seems you have done a lot of background research and sometimes its hard to understand some of this stuff without just bashing you head against the wall (or asking in a forum such as this). I myself am a long time lurker (first time poster actually), and hopefully I'll be able to answer some of your questions.


1. My first question is, what is Haali Media Splitter and what does it do?


Haali Media Splitter is a DirectShow component (I know you asked what DirectShow is below, I'll get to that in a moment) which, for all intents and purposes, demuxes a file and provides the streams (video, audio, subtitles, etc.) to other components that can decode them and display (or send it to your video card for audio) them. I believe it was one of the first MKV demuxers, (someone please correct me if I am wrong) but it currently supports many more formats (see the Haali Media Splitter Homepage (http://haali.su/mkv/) for more info. The icon in your systemtray is there to let you know that it is loaded and in some cases you can change certain options on the fly, like for player programs that don't internally support multiple audio streams.


2. Following on from the previous question, why is the FFv icon there? Is it because ffdshow is decoding the .m2ts file? Is it because ffmpeg mt is decoding the .m2ts file? Is ffdshow the same thing as ffmpeg mt? What exactly is ffdshow? What exactly is ffmpeg mt?


Many questions in one! The FFv icon is there to let you know that the video stream (FFv for video) is being decoded by ffdshow. ffdshow is a port of the ffmpeg decoders to Windows (through DirectShow). ffmpeg-mt is a multithreaded version of the ffmpeg program (meaning it can take advantage of multiple cores on your CPU for decoding). I believe there are builds of ffdshow which include ffmpeg-mt. So they are related, but not exactly the same.


3. I use a program called Win7DSFilterTweaker_4_1. According to its description, it allows you to change the preferred DirectShow decoding filters in Windows 7. I'm running Windows 7, but I've no idea what DirectShow is or what a decoding filter is. I assume a decoding filter is what renders a h.264/mpeg2/vc-1/etc video stream. Is this correct?


DirectShow is the Windows framework for displaying video. This framework includes components called "filters." Each filter takes the job of one step of the task of displaying video on the screen. For example, one filter might open a file from the hard drive (a different one might load data from a webserver, for example). Another might demux the file (like Haali Media Splitter). A third might decode the video into an uncompressed format. Yet another might have the task of physically displaying it on the screen.

All these filters are hooked together in a chain (so you'll hear about DirectShow "filterchains" often). In Windows 7, there is a slightly updated framework of decoders which, by default, will always supersede any other filters on your computer for playing in players such as Windows Media Player. Win7DSFilterTweaker_4_1 lets you override these settings so you can use different decoders (such as ffdshow) instead of the Microsoft supplied ones.


4. ...I understand the concept of containers, muxing and demuxing...


Saves me some typing then :P.


5. Regarding Win7DSFilterTweaker_4_1, what settings should I be using for H.264, VC-1 and MPEG-2? ...As I understand it, ffdshow DXVA means the video decoding (ie, video playback) is computed on the videocard, so does that mean that the only benefit to setting it to ffdshow DXVA for H.264 and VC-1 is there will be less load on the CPU?


Hmm, there are no specific "correct" settings for a program like this. Its up to you to decide which decoders you want to use for different material. As I dont personally use this program, (or Windows Media Player) I'm not sure what you have to set it to in order to force it to always use something like ffdshow.

DXVA is an interface to allow video to be decoded by special decoder chips on video cards. So yes, if DXVA is enabled (and a video clip is within the specs to allow DXVA to be used) there will be less load on the CPU.


6. I understand frame rates, the concept of 2:3 pulldown and the concept of ivtc. I understand the concepts of CPU cores, SMT and threads. I understand the difference between YUV/YV12 and RGB colour.


Good! Its a complicated topic.


7. When I used to encode a .m2ts file (with a H.264 video stream) in MeGUI, I used to get a message at the 'enqueue' stage (ie, when adding the file to the encode queue) that said the source file was not in YUV colour format and MeGUI would ask me if I wanted to add the 'ConverttoYV12() function to the Avisynth script. What I don't understand is, when I used MediaInfo on the file it said that the colour space was already YUV (which is, correct me if I'm wrong, the same as YV12). If this is true, then why was MeGUI not getting the source in YUV colour?


From what I remember, certain decoder filters sometimes convert data from the native YV12 color space to RGB before sending it to the Renderer component (which is what displays it on the screen). Since it has to be converted to RGB to display on the screen at some point, perhaps they wanted to make it "easier" on the Renderer component (this is a known issue with some decoders used through DirectShow). Since you were converting from a YV12 format to a YV12 format, MeGUI needed to tell Avisynth to convert it back to YV12 before the encoder would accept it. Multiple color space conversions like this can lead to a loss of quality.


This used to happen before I knew about Win7DSFilterTweaker_4_1, and I resolved this by using Win7DSFilterTweaker_4_1 to change the H.264 option from 'Microsoft' to 'ffdshow DXVA'. I noticed that, after I did this, the FFv icon started showing up in the system tray whenever I opened the .m2ts file. I can only assume this is because ffdshow was now being using to decode the video file, but why did making this change solve the ConverttoYV12() problem?


This happened because ffdshow did not throw in the extra color space conversion to RGB, like the previous decoder did. Therefore video in YV12 was being passed directly to the encoder without any unnecessary conversion.


8. Following on from the previous question, I recently came across a Blu-ray that uses a YUV MPEG-2 video stream. When I added it to the MeGUI queue, I received the 'ConverttoYV12()' message again. So, as before, I used Win7DSFilterTweaker_4_1 to change the MPEG-2 option from 'Microsoft' to 'ffdshow' (there was no ffdshow DXVA option for some reason - anyone know why?). This time, however, it didn't fix the problem. I still received the ConverttoYV12() message.


I'm not sure about this one. You can try using FFMS2 (ffmpegsource2) which is a plugin for Avisynth which also uses the ffmpeg decoders to provide video to Avisynth. Since it is separate from DirectShow, it isnt susceptible to some of the same problems that DirectShow decoders have.


9. What is a 'frameserver', what are some examples of frameservers, do I need a frameserver and which is the best frameserver?


A frameserver is basically exactly what it sounds like. Its a program that serves individual frames to another program. They are most often used to link two separate programs (such as a decoder and an encoder) that otherwise do not have the capability to talk to each other. You can call a program like Avisynth a frameserver since it 'serves' frames to other programs (such as the x264 command line tool). There are other programs that can be used as frameservers (such as VirtualDub) but each one has very specific uses, so there is no 'best' frameserver. Just one that fits the job you require of it. Do you 'need' one? Again, that depends on what you are trying to do.

I hope I have been of some help. I'll try to keep an eye on this thread in case any more questions come up, or I need to correct any inaccuracies.

[pwnsweet]

I've been using the internet for about 11 years. In that time I've researched everything from computer hardware, overclocking and laptops to HDTV's, modding and all things electronic. In all that time, I can only remember a few scarce moments when I've asked a question on a forum and even fewer moments that my question was answered as well as you've done here. In fact, it has happened so rarely that in the last few years I've basically given up asking anything on forums, resorting to trying to find the answers to my questions through trial and error (when applicable) or extensive research instead. Simply saying 'thank you' as a way to express my gratitude for your time spent helping me without adding this preface would be an understatement to a degree that borders on blasphemy, however now that you're aware of my history, I will say 'thank you' and hopefully it will be interpreted differently to the way it would have been interpreted without this preface.

So, thank you for your answer nhakobian. It is very much appreciated.

I do have another question. What is libavcodec?

[Gavino]

What is libavcodec?
http://en.wikipedia.org/wiki/Libavcodec

[pwnsweet]

Thanks Gavino. Now that I've done some reading up on libavcodec, I have some more questions!

So just to confirm that I'm understanding everything properly so far:

- Haali Media Splitter is a demuxing filter and it directs the video and audio stream to a decoding filter
- ffdshow is a decoding filter that can accept these streams and decode both video and audio if required (I've been reading and apparently it can encode as well?), and
- ffmpeg and libavcodec are two examples of decoders that are within ffdshow that will decode these video streams depending on which of these two decoders is set in the 'video decoder configuration' menu of ffdshow.

If that's correct then I have more questions so I've gone ahead and updated the OP with Q10-Q15

[pwnsweet]

Added Q16 and Q17 to the OP

[sneaker_ger]

10. There is no difference for progressive content, as they all follow the standard. There can be differences though in terms of built-in deinterlacing and/or post-processing, but these are not strictly speaking part of the decoder.

11. ffmpeg and libavcodec are more or less the same thing. As long as you don't experience any problems, both are fine. For H.264 choose "ffmpeg-mt", as it will make better use of multiple cores.

12. see 10.

13. Experiences differ and I see a lot of people switch to "LAV Filters" lately. But as long as you don't have any problem, either of them is fine.

14. They both access the same part of the chip for decoding, they just use different APIs. While DXVA decoders are often limited to playback, "CUDA" has the advantage in that it can be used just like a software decoder, e.g. for transcoding Blu-Ray to mp4. (There are attempts to do this with DXVA, too)

15. Important for image quality is the renderer which converts the colorspace and does the scaling, and maybe a good deinterlacer. madVR and LAVCUVID are very good for this.