YV12: mpeg2dec, mpeg2dec2 and mpeg3dec developers

[Marc FD]

>I guess yours does not handle HDTV streams so I'm going to have to do MPEG2DEC2, but I'm lazy.

it's based on save-oe (MPEG2Dec2) and there's pid stuff in it. i guess it should be able to handle HDTV streams. if not, just say what you need, i'll add it ^_^

[MaTTeR]

Originally posted by vlad59
Do you mean that the presets don't work anymore ?????

No, sorry that's not what I meant. Presets appear to work fine for me so far. In your readme(documentation) file it shows that all presets use the same parameters;) Copied and pasted here- 2 - Parameters sample

I build the following presets to make things easier :
Convolution3d (preset="movieHQ") // Movie Hi Quality (good DVD source)
is an alias for Convolution3D (0, 32, 128, 32, 128, 10, 0)
Convolution3d (preset="movieLQ") // Movie Low Quality (noisy DVD source)
is an alias for Convolution3D (0, 32, 128, 32, 128, 10, 0)
Convolution3d (preset="animeHQ") // Anime Hi Quality (good DVD source)
is an alias for Convolution3D (0, 32, 128, 32, 128, 10, 0)
Convolution3d (preset="animeLQ") // Anime Low Quality (noisy DVD source)
is an alias for Convolution3D (0, 32, 128, 32, 128, 10, 0)
Convolution3d (preset="animeBQ") // Anime Bad Quality (???)
is an alias for Convolution3D (0, 32, 128, 32, 128, 10, 0)

[trbarry]

>"I guess yours does not handle HDTV streams so I'm going to have to do MPEG2DEC2, but I'm lazy."

it's based on save-oe (MPEG2Dec2) and there's pid stuff in it. i guess it should be able to handle HDTV streams. if not, just say what you need, i'll add it ^_^

Marc -

Okay, I guess that means the source isn't available. I really hate to lose my SSE2 speed advantage. :(

But I only tried it a couple times with ATSC streams but got an error on both files so I switched my YV12 testing to DVD material. Since I didn't have source I didn't try to debug it much. I'll go test some more and check and see if it is just something simple in the d2v file and let you know.

Does your code read the PID numbers in the d2v file? Or is that part of the code the same also? In doubt, my version source is still at:

www.trbarry.com/MPEG2DEC2.zip

- Tom

[Marc FD]

>Okay, I guess that means the source isn't available. I really hate to
>lose my SSE2 speed advantage.

i removed the sse2 stuff, because it crashed for P4 owners. i'm not sure your sse2 MC is working. BTW, sse2 iDCT works, and it's more important than MC (speed-wise)

>But I only tried it a couple times with ATSC streams but got an error on
>both files so I switched my YV12 testing to DVD material. Since I
>didn't have source I didn't try to debug it much.

It's transport packet stream compatible.

>I'll go test some more
>and check and see if it is just something simple in the d2v file and
>let you know.

>Does your code read the PID numbers in the d2v file?

yes, with Stream_Type=2,[..],[..] and MPEG2_Transport_PID=VideoPID,[..] is recognised. i think it should work. i hope i didn't broke something.

[int 21h]

Why isn't the source to mpeg2dec3 available?

[vlad59]

Originally posted by Marc FD
It'll ONLY output in YV12, but i'll add the internal YV12->YUY2 convertion (for debugging, and to keep compatibility with MPEG2Dec. MPEG2Dec3 would still be able to give the same results than MPEG2Dec. (maybe not exactly, because of the ssemmx Motion Compentation)


If I remember well (it's been a long time) the ssemmx MC code I send to you has the same accuracy as the original MMX code. I had also written some Fast&dirty one (a little faster less than 1% but not accurate at all). So If you kept my code there shouldn't be not problem.


@MaTTer

That's the problem when you make the tests on the laptop and make the release on the _normal_ computer -> some copy/paste are not automatical ;) ;) ;)
I'll fix that soon.

[Marc FD]

>If I remember well (it's been a long time) the ssemmx MC code I send to
>you has the same accuracy as the original MMX code. I had also written
>some Fast&dirty one (a little faster less than 1% but not accurate at
>all). So If you kept my code there shouldn't be not problem.

yes you're right. i've even made tests between MPEG2Dec and MPEG2Dec3 and the results were 100% the same (and the speed almost the same too, maybe about 5-10% speedup ^^)

>Why isn't the source to mpeg2dec3 available?

no fear. i know it's GPL code, if you WANT it, you'll get it. but if i don't give it, it's because i'll need a _lot_ of time to clean what i've done and i think it's a waste of time. i prefer concentrate on usefull coding. if someone complain, no problem, i would stop to release it ^_^.

BTW, i've finished Focus2 - YV12. i'm gonna release it soon.

[int 21h]

Originally posted by Marc FD
no fear. i know it's GPL code, if you WANT it, you'll get it. but if i don't give it, it's because i'll need a _lot_ of time to clean what i've done and i think it's a waste of time. i prefer concentrate on usefull coding. if someone complain, no problem, i would stop to release it ^_^.


I'd prefer you just obey the GPL like everyone else does (trbarry, neuron2, sh0dan, etc.) No one around here cares how messy your code is, and in fact, some might even help you clean it up a bit.

Not like I'm asking for an entirely documented project, pseudo-code, algorithms, and incode comments, just that you obey the original intent of the code (So that if Tom indeed wanted to investigate the issues with transport streams in the code, he could).

[trbarry]

Marc -

I don't mean to be pushy but probably the only reason there IS an MPEG2DEC3 with all these nifty features is because there was open source to MPEG2DEC2, which wasn't cleaned up either.

And the ony reason there was an MPEG2DEC2 was because there was available source to DVD2AVI and MPEG2DEC that a whole bunch of people contributed to, making this sort of thing possible.

I'm not the original author of either of them but I'd like to see them continue to evolve. You are doing a quite admirable job in this area, but there is still only one of you. If you made it available then, like me, you would probably find that open source is a bit like karma ... what goes around, comes around. You'll find that other people add useful functions to it also.

And I don't think you are in a position to easily test and debug either ATSC/HDTV or SSE2 support but it would still be a shame if this caused those functions to disappear.

And frankly, I'm still too lazy to go code YV12 MPEG2DEC2 output routines if I know there are already functional ones based upon that code. ;)

How about if you post those output source functions and leave the rest 'til it's cleaned up and convenient? Possible? No demands, just a suggestion.

- Tom

[Marc FD]

>I don't mean to be pushy but probably the only reason there IS an
[...]
>And I don't think you are in a position to easily test and debug
>either ATSC/HDTV or SSE2 support but it would still be a shame if this
>caused those functions to disappear.

i think i wasn't clear. i'm addicted to opensource devellopement. no problem with that. if you want the source, just email me, i'll give it to you ASAP. my problem is packaging.

>And frankly, I'm still too lazy to go code YV12 MPEG2DEC2 output
>routines if I know there are already functional ones based upon that code.

you're are overestimating me. YV12 support is a 10 min hack.

>How about if you post those output source functions and leave the rest
>'til it's cleaned up and convenient? Possible? No demands, just a
>suggestion.

i think the best way is :
you email me, i send you the code, you test all sse2 parts of the code, make a sse2 version of the new MC, HDTV support, ect...
and you (please ^^) say me what's changed, so i can update it, and every user could benefit from it.

my only fear is to have 2 differents versions.
but i think opensource devellopement can be very productive.
i know well MPEG2Dec's code, you too, i think we can easily work together on the same project.

Regards,
MarcFD

PS : if someone want the code, just e-mail me. i don't think it's really interessing. BTW, when it would be stable, i'ld clean the code and package the source code. i just don't like too lose 50% of my time in code packaging of beta versions.

[wanton]

I just want to end this stupid YV12 mpeg2dec.dll thing because it's not possible to do. Only format that would work would be frame encoded mpeg2 file like svcds if you use force film option. Because most atleast anime dvd's have field/frame encoded frames meaning chroma is halved in field and frame mode diffrently in field mode it's halved field based and in frame frame based. and YV12 in avisynth is frame based. Then another problem is pulldown if you do normal output from mpeg2dec from material with pulldown added with flags mpeg2dec makes virtual frames using flags so if you do it in YV12 you would get quality loss because you would have to blend chroma and it would look shitty. Same thing with IVTC when you combine frame by field basis YV12 is shitty format. So only format YV12 would be any good for would be progressive frame based encodes without pulldown. Even progressive looking material could have been encoded in field format Looks progressive but have been encoded with field based.

[Koepi]

wanton:

you might want to do some background checks before posting such an misleading, wrong crap. Sorry pal, but the info you're "relying" on is wrong.

If your assumptions were right then mpeg2 couldn't be YV12 colour space internally - but it is.

Now please do some research, let us non-anime encoders our progressive YV12 fun and stop rumbling around. Read the forum rules, you sure broke no. 4.

*scratching head*

Koepi

[trbarry]

wanton -

Be aware that Avisynth already maintains flags for each clip saying whether it is frame-based or field based. And it is mostly only the deinterlace, IVTC, and conversion functions that will care.

It is true that Avisynth considers the entire clip to be either one or the other but if that becomes a problem for mixed clips we can pass along additional flags from any YV12 supporting versions of MPEG2DECx. Frankly I would like to have them anyway because of the guidance they could provide to IVTC type functions.

You might also consider that it is the job of a deinterlace or IVTC function to convert a clip to all progressive frames. Since these functions should invariably be run first before other filters you can figure that most Avisynth functions can then be operating on frame based material. And as I mentioned above (I think this thread) those IVTC/deinterlaced functions should be sure to always set the AssumeFrameBased flag for clips they process.

We've probably glossed over other details also but nothing that I do not believe is solvable. And I still believe that handling things in their native YV12 format will give both faster and more precise results. For instance I think there can be much greater chroma precision if deinterlace/IVTC is done before any YUY2 color conversion since frame based clips can be created. And we've already seen that many operations may be written to be faster.

So sure there are issues to be dealt with but with much of our input material now coming from MPEG2/YV12 in the form of DVD's, Tivo's, and (increasingly) HDTV it would be silly not to explore this.

I think at worst it will turn out that some clips can not be processed optimally using both YV12 and DirectShowSource, but I don't consider DirectShowSource all that reliable anyway. But I'm pretty confident that at least the DVD2AVIx/MPEG2DECx YV12 path can be made quite functional and eventually better for many purposes than either YUY2 or RGB.

- Tom

[scmccarthy]

WOW Tom, that makes a lot of sense.

I think it is intuitively obvious that if we want to reencode mpeg-2 into mpeg-4, becuase it compresses better, since they both use the same color space it is better to stay in that color space because color space conversions are lossy. You also point out that some filters are actually easier to write in YV12 than YUY2. And I think that filter writers who have already made the transition from RGB to YUY2 will have no problem learning YV12 provided the benefits are worth it and they are. Meanwhile it is important to have the best conversion filters possible so filter writers can write in the color space of their choice with minimal impact on quality.

I am trying to write a filter in c++ using just cl.exe and link.exe that converts from YV12 to YUY2 by resizing the luma rather than interpolating the chroma. I'll post the code in a new thread later, but meanwhile, since you mentioned flags, I wonder if you know whether Avisynth 2.5 has some flag or structure member that needs to be set saying color space is now YUY2 instead of YV12? Or to change the size, since the vertical resolution gets reduced by a factor of 2?

The new YV12 Invert and my code compiled properly, so my primative development environment does work. Don't accuse me of going off topic, this question is about Avisynth 2.5 flags and structures as it relates to YV12. I'll ask about my code in a new thread.

[Marc FD]

Hi ^^

i'm dead. i've coded 5 hours to get MPEG2Dec3 v0.9 ready (but not clean enough code-wise :( )
and i've spend 2 hours on the documentation, because my filter always lack of it, and i hope my efforts would be appreciated.

i'm gonna open a new thread for MPEG2Dec3 v0.9 it's avisynth 2.5 ONLY compatible, able ouput native YV12, and to convert to YUY2/RGB24 (interlaced or progressive) , to Separate YV12 Fields (only topfirst yet), and some other stuff.

@tom

i'll give you the code ASAP. but i think it couldn't today : i really need to sleep ^^.

[Wilbert]

@tbarry,

If you are going to modify mpeg2dec3 you modify it (and dvd2avi) such that the AC3 stream is passed through (if it is present of course, since there's no mp2 support yet).

[int 21h]

Originally posted by Wilbert
@tbarry,

If you are going to modify mpeg2dec3 you modify it (and dvd2avi) such that the AC3 stream is passed through (if it is present of course, since there's no mp2 support yet).

I thought AVISynth had no compressed audio support yet anyways?

[hakko504]

Compressed audio support was introduced in v2.04. List of changes and latest CVS release
is here (http://cultact-server.novi.dk/kpo/avisynth/avs_cvs.html)

[Wilbert]

I thought AVISynth had no compressed audio support yet anyways?

You need to be more around on this forum! Since v2.04 there is compressed audio support (DivX/Xvid with mp3 audio VBR/CBR).

In v2.5 there is multichannel support (DivX/Xvid with AC3 audio VBR/CBR).

and there are more plans ...

[sh0dan]

AviSynth doesn't have INTERNAL compressed sound, and probably will never have - I think this is what int 21h is talking about.

All audio still needs to be decompressed, just as images have to.

[int 21h]

Originally posted by Wilbert
You need to be more around on this forum! Since v2.04 there is compressed audio support (DivX/Xvid with mp3 audio VBR/CBR).

In v2.5 there is multichannel support (DivX/Xvid with AC3 audio VBR/CBR).

and there are more plans ...

I get around quite enough thank you.

And as sh0dan confirms, I meant passing through audio similar to WavSource() functionality. Instead of crutching mpeg2dec to pass AC3, work should be done on an AC3Source() filter, or maybe a general AudioSource() filter.

[sh0dan]

Originally posted by int 21h
Instead of crutching mpeg2dec to pass AC3, work should be done on an AC3Source() filter, or maybe a general AudioSource() filter.

The most obvious way (IMO) would be for someone to create a link to besweet.dll. It offers decoding of all important standards.
A besweet input/output plugin would be VERY helpful for all cases, and should be much easier with the 2.5 API.
I would love to help anyone up to the task, if they get problems with the AviSynth part.

[int 21h]

a BeSweet I/O plugin would more or less be something like..

BeSweet("audiosource.wav/ac3/etc",500 parameters here)

or are you talking about trying to encode the stream of an input given, like the soundtrack of an avi file or MPEG file. In its current incarnation, its impossible to use BeSweet.dll as a simple decoding tool because it always outputs its encoded result into a file (so you couldn't just pass a sample of PCM to it and get back a sample of MP3 and pass that through Avisynth to the destination environment)

It seems like it would be so much more useful to have the functionality inside of Avisynth so that you could compress/process outside of Avisynth, and the support of the input of compressed audio but lack of support for the output of compressed audio seems .. :confused:

[sh0dan]

OK, let's just start all over then - this is why I would like something like besweet:

I would very much like for people to be able to do advanced editing, even if their source is from a DVD. That also means that audio has to be put alongside video. For example:

Mpeg2dec2("poelle.d2v")
audiodub(AudioSource("poelle.ac3"))
trim(505,-10000)
l_ch = getchannel(1)
r_ch = getchannel(2)
mergechannels(l_ch,r_ch).normalize().ssrc(rate=44100)


So the user gets out two-channel, completely in-sync audio. As MP3 or just WAV for starters.

What would be the best solution in your opinion?

I'd really wish I has the time to get in some great audio functionality, like ssrc, SuperEQ, Boost (sound compression). But I cannot do the code for it until 2.5 is out.

[Nic]

I started it & then got completely sidetracked. At present, as far as I know, besweet.dll doesn't let you have access to the output frames (i.e. it gets you to specify an output file then you use *_Burst to send the input packets).

That would need to be changed before a besweet plugin can be made....

-Nic

[int 21h]

Originally posted by sh0dan
OK, let's just start all over then - this is why I would like something like besweet:

I would very much like for people to be able to do advanced editing, even if their source is from a DVD. That also means that audio has to be put alongside video. For example:

Mpeg2dec2("poelle.d2v")
audiodub(AudioSource("poelle.ac3"))
trim(505,-10000)
l_ch = getchannel(1)
r_ch = getchannel(2)
mergechannels(l_ch,r_ch).normalize().ssrc(rate=44100)


So the user gets out two-channel, completely in-sync audio. As MP3 or just WAV for starters.

What would be the best solution in your opinion?

I'd really wish I has the time to get in some great audio functionality, like ssrc, SuperEQ, Boost (sound compression). But I cannot do the code for it until 2.5 is out.

I am of the same general opinion, except I believe the output format and encoding process should be left up to the user. Decoding AC3 to Wav is a good idea, but in my mind encoding to mp3 just seems like a bad idea, also offering the pass through of AC3 would be convenient.

In my mind, Avisynth is an editing framework, and as such it should not be reliant on any specific encoding technologies because of the pace at which those technlogies evolve. (Consider 2 years ago what we encoded MP3s with)

All of this really became relevant to me when I started helping out with AVS2AVI because only having a WavSource() input filter for audio severely limits any program that relies on the Avisynth interface. In my mind AudioSource would have arguments, allowing you turn decoding on/off, and AudioSource ideally would not be reliant on third-party .dlls (using internal decoding like AC3Dec, which could probably use some work), however, you could use the straight azid.dll because its calls work as follows:

//
// Decode one frame of ac3
//
// Decode the ac3-frame in 'ac3data' into pcm samples in 'pcm'. The output
// samples are interleaved according to the setting 'speaker_sequence'.
// E.g for stereo the first sample (pcm[0]) is left channel, sample 0, and
// the second (pcm[1]) is sample 0, right channel, etc. The float value
// range of the pcm samples are -1.0 to 1.0.
//
// Returns: The result of the operation. Use the following macroes to check
// if the operation is successful or not:
// AC3_OK() The operation was successful
// AC3_FAILED() Some serious error has occurred and the
// decoder program should quit.
// AC3_PRODUCEDOUTPUT() The decoder has produced data, i.e. the
// data in 'pcm' is updated and contains
// valid data.
//
int ac3_decode(u32 *ac3data,f32 *pcm);


and you could still deliver a single sample of pcm for the final encoding application.

[sh0dan]

IMO all import/export of compressed audio formats should never be an internal AviSynth functionality - just as MPEG2 decoding also shouldn't be.

All of my suggestions were for plugins. I don't think any of these belong inside AviSynth, but they should be separate plugins. Therefore depending on external dll's is not that big a problem, IMO.

[int 21h]

I admit that makes more sense, I'm not familiar enough with Avisynth framework to know how to make a plugin like AC3Source(), but maybe if I get some time I'll look... it can't be all that different from the principles in Mpeg2dec except that you're working with audio.

[sh0dan]

Basicly, the only difference is that you extend GetAudio instead of GetFrame.

You can see how to handle audio in audio.cpp/audio.h in the AviSynth sources - it's really easy, if you have worked with samples before.

[int 21h]

I'm somewhat confused on how you need to offer seeking capabilities. I'm expected to provide a GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env), Azid only provides a function to read 1 frame into memory, and it appears to be sequential access only (i.e. I can't specify what frame or sample I'd like to retrieve, but if I built my own reading function I suppose I could supply it the frame to decode via the Decoding function), even if I do supply my own reading function, there will still be times that you have to loop forward to seek in the file right?

[sh0dan]

Yes, assuming that you get sequencial reads isn't very safe.
You will probably have to include your own buffering, so that it can handle requests that split an AC3 frame (which is very likely).
So you have to buffer the remaining samples, until the next read request, where they are needed.
Another way would of course be to decode the same frame again - probably easier, since it is the typical case, and probably nothing worth noticing performancewise.

As a side note, EnsureVBRMp3Sync() gives the preceding filter 100% assurance that it is read sequencially. If the user rewinds, ALL audio is decoded from frame 0 = very slow.

[DSPguru]

Originally posted by Nic
I started it & then got completely sidetracked. At present, as far as I know, besweet.dll doesn't let you have access to the output frames (i.e. it gets you to specify an output file then you use *_Burst to send the input packets).

That would need to be changed before a besweet plugin can be made....

-Nic i might find the time on weekend to supply this.

[Nic]

No rush Dg, im far too busy, ill and very irritable :)

@int21:
When I started writing to think about writing the BeSweet audio plugin
(which I think I aptly called BeSynth in the VC6 workspace ;) ) I wrote a little skeletal plugin that does the same as WavSource (but reads the data in raw from the Wav file, rather than use the AVI API).
Might be a starting point if your short on time, if you want?

-Nic

[int 21h]

Sure, PM me the details :D

[wanton]

koepi - Sorry that calling mpeg2dec yv12 conversation stupid got on yer nerves but it wasn't offending anyone except it seems probably you. I didn't mean to do that. mpeg2 is yv12 but every frame can be in diffrent format meaning way chroma is handled like i said. Reason example why DVD2AVI author did DVD2AVI was because flaskmpeg decoded interlaced frames wrong http://arbor.ee.ntu.edu.tw/~jackei/dvd2avi/interlace/
Here in that page you probably better explanation about what i mean.

But because avisynth only has interlace flag per clip not frame and i'm not even sure avisynth supports interlaced YV12 currently. Interlaced chroma would mean first line in chroma would have chroma for line 0 and line 2 if it is progressive it's for line 0 and line 1 and so on. trbarry was right about tivo stuff because it doesn't have pulldown flags so it doesn't have to comb frames so it should work and ivtc it would work because it's probably field encoded but there are adaptive frame/field encoders (field for high motion and frame for low motion scenes) so it could mean for some frames you would still have to blend chroma. It's just that doing all this doesn't speedup so much and causes probably lot of problems because you would have to know what you can or can't decode properly with mpeg2dec to yv12 and ivtc would only work without quality loss in chroma for field encoded frames since chroma is divided per field. So doing YUY2 to YV12 after ivtc + deinterlace then running filters in YV12 should be good since final encode is gonna be encoded progressive mpeg2/4 anyway so there should be any quality loss doing this.

[int 21h]

Again you're not researching anything before talking your talk. From the Avisynth 2.5 alpha page (developer's notes)
From http://cultact-server.novi.dk/kpo/avisynth/avisynth_alpha.html
YV12 interlaced chroma
YV12 handles interlaced chroma differently than compared to YUY2, since YV12 only contains chroma information for every second line. To enable interlacing, chroma is stretched across two luma lines in the same field! That means that luma and chroma isn't directly mappable to lumaline/2 and lumaline/2+1 as framebased images.


line 0: Chroma for interlaced luma lines 0+2 line 1: Chroma for interlaced luma lines 1+3 line 2: Chroma for interlaced luma lines 4+6 line 3: Chroma for interlaced luma lines 5+7...etc!

This is something that deinterlacers and similar programs need to take into consideration.Other filters might rely on the use of Separatefields() and Weave(), to produce framebased images. You can use the VideoInfo.IsFieldBased() to check your source, and maybe decide to throw an error, or shift to another processing mode.

If your video is fieldbased your vertical resolution (height) must be divideable with 4, otherwise AviSynth will not create a new VideoFrame, but will throw an error.

[Marc FD]

you should take a look at MPEG2Dec3. it's outputting YV12.
try SeparateFieldsYV12 or YV12toYUY2(interlaced=true) on interlaced MPEG2 sources.

Avisynth is the more powerfull existing video framework. don't compare it to FlaskMPEG please.

Thx.

[scmccarthy]

If the YV12 to YUY2 conversion only works for frame based clips, then SeparateFields of SelectEven/Odd then stacking them, doing the conversion, then Weaving, would allow you to use the frame based version of the conversion. Soon, if not already, it will work for fields. Maybe this is already clear to everybody; but I like to state the obvious once.

[trbarry]

Wanton -

Maybe there is some confusion here about how YV12 works in MPEG2. Possibly that confusion is mine, but if so then if I write it down here someone else can point it out.

Please pardon my oversimplifying here. I'm thinking out loud.

I'm using the "Video Demystified" descriptions here. In my pictures I'll call the top line line 1. Be aware that many/most video folks tend to number them from one, except in maybe in DScaler where we've usually called it 0.

In MPEG2 4:2:0 there are values representing Y, U, and V, where Y is the luma (brightness) and U,V represent chroma (color) values. I'll just call the U and V components C for chroma, since in MPEG2 they will represent 2 different values at exactly the same sample points on a screen.

But there are 4 times as many Y components as C components, twice as many in both the horizontal and vertical directions. Each Y component represents the Luma value at some sample point on the screen. And each C component represents chroma value at some point on the screen. Much of the confusion in MPEG2 4:2:0 color space results from the fact that the Y and C components result from values (originally measurements) at different points on that screen, no 2 at the same point.

If the luma represents sample points on screen lines 1,2,3, ... then the MPEG2 chroma will represent sample points on lines 1.5, 3.5, 5.5, etc. None of these are real screen raster lines. The sample points occur between the real screen lines and when displaying things you eventually have to interpolate to get the correct values. And only half of those possible half-lines are actually represented with chroma values.

For instance, ignore how YV12 is stored in planes and let's just think about which points on the screen are represented by the data. For a progressive frame the following sample points might be represented:


Line 1 : YYYYYYYY ...
Line 1.5: C C C C ...
Line 2 : YYYYYYYY ...
Line 2.5: <- not present
Line 3 : YYYYYYYY ...
Line 3.5: C C C C ...
Line 4 : YYYYYYYY ...
Line 4.5: <- not present
Line 5: : YYYYYYYY ...
Line 5.5: C C C C ...
Line 6: : YYYYYYYY ...
...


So even in progressive, if you want to find all values of chroma for all the pixels you want to display you have to interpolate between the various chroma sampling points you have been given. For instance to calculate the croma values for line 4 you will need to use ratios of chroma values from (at least) lines 3.5 and line 5.5.

Okay, cool, we have code to do this. Now what happens if we have an interlaced clip? Then even though we are given a single frame it may contain data from two fields which represent 2 different points in time. So if we were to combine chroma data from lines 1.5 and lines 3.5 we would be blending chroma from 2 fields. And we know this is a bad thing, possibly causing artifacts since those fields change over time.

So it looks like MPEG2 4:2:0 for interlaced frames ("Field Based") is stored like the following (assumes top field first):


First Field Second Field
Line 1 : YYYYYYYY ...
Line 1.5: C C C C ...
Line 2 : YYYYYYYY ...
Line 2.5: <- not present
Line 3 : YYYYYYYY ...
Line 3.5: C C C C ...
Line 4 : YYYYYYY ...
Line 4.5: <- not present
Line 5: : YYYYYYYY ...
Line 5.5: C C C C ...
Line 6: : YYYYYYY ...
Line 6.5: <- not present
Line 7 : YYYYYYYY ...
Line 7.5: C C C C ...
...


So now look what happens. If we need to calculate the chroma value for pixels on line 4 we can no longer use the values from line 5.5 because they come from the wrong field and possibly represent the wrong point in time. So we would have to use a ratio of chroma values from line 3.5 and line 7.5. This keeps us using only the correct field but now we are vertically much further apart and the calculation will be much less accurate, losing color detail. However that is the way it is currently done as long as MPEG2DEC2 just returns YUY2 data for field based frames.

But here is the point where I may be wrong and someone can jump on me for it.

I believe that for both the progressive and interlaced cases above the croma sample point values represent EXACTLY THE SAME points in space, just not necessarily from the correct fields, or points in time. But if this is true then, to the extent we can IVTC or deinterlace the frames first, we can then do the more precise color conversion that can be used with progressive frames.

Now it remains to be seen in real life whether we can do this, and for which real life clips we will need the MPEG2 flag data from MPEG2DECx.dll in order to do it. But, again, I'll wager that it is doable and that we will see an advantage from it.

So, what did I overlook? ;)

- Tom

[vispgraedde]

Originally posted by int 21h
Again you're not researching anything before talking your talk. From the Avisynth 2.5 alpha page (developer's notes)

Hmm... Your own quoted post would be more interesting if you could relate the information you posted to the information in the link that Wanton posted.
If the information can be shown to be equal, then all is good.
But if not, I'd personally trust what the dvd2avi author says more than what the avisynth developer says.

[scmccarthy]

@trbarry

Please check out my one and only filter ConvertYV12ToYUY2VerticalRecuceBy2() for Avisynth 2.5 alpha. It is great as an example code that demonstrates the inner workings of YV12 that you are talking about. I do remember a while back you asked for a a YV12 version of Invert. This is like that execept in uses a little more of the new YV12 functionality.

Try it and I think you will be satisfied that it demonstrates the difference between YUY2 and YV12. The output certainly looks right.

It is not meant to work with interlaced material. I have not even tried it with interlaced material yet.

[trbarry]

But if not, I'd personally trust what the dvd2avi author says more than what the avisynth developer says.

I guess that depends upon which application you are talking about.

But it may not matter how you prefer to convert YV12 if you can avoid doing so. And to the extent we can pass only progressive (or all field based) YV12 frames to Xvid we can do just that.

- Tom

[vispgraedde]

For clean progressive stuff, it should mean a speedup if you can use only YV12 throughout the whole process, but since almost no dvd I have is progressive, it's more important to me to have something that is garanteed to work than something that might work (if it works out in the end, all is good, but until then...)

[trbarry]

Well, it seems it should work whenever you have progressive material.

And it seems it should work whenever you can use DVD2AVI/MPEG2DECx force film.

And it seems it should work with all interlaced video when you intend to keep it that way.

And it seems it should work whenever you are going to deinterlace video source with TomsMoComp, even if there are some progressive frames in it.

And offhand I think we can probably make it work with things like DeComb and GreedyHMA for most material but there are still some issues I haven't tried to work out yet.

And we can already predict that at least some Japenese anime will find a way to break it somehow. ;)

But it looks like it should work at least some of the time. :)

- Tom

[Marc FD]

(for all the suspicious guys about YV12 and interlacing)

> And we can already predict that at least some Japenese anime will find a
> way to break it somehow.

guess what : i encode ONLY japanese anime.
and better : often after NTSC->PAL convertion.
a real nightmare. (a sort of worst-case encoding ^^)

i don't see what is the problem with YV12 ?
i've programmed a bob filter in YV12, and it works very well.

you can trust a guy who worked a lot on MPEG2Dec and who's encoding only interlaced streams, when he says that YV12 is a pleasure to code, and has only advantages.

BTW, YUY2 is not forbidden in Avisynth 2.5 ^__^

[SansGrip]

you can trust a guy who worked a lot on MPEG2Dec and who's encoding only interlaced streams, when he says that YV12 is a pleasure to code, and has only advantages.

I disagree. This may be true if all you need to do is iterate over each component individually while applying some processing to them. But as soon as you get interdependence between the components I find a packed format much nicer.

It's a question of speed of execution/speed of development. YV12 is undeniably faster at runtime, but I find it to be at least twice as much typing, which means twice as many typo bugs. It also means three times the pointers and at least two times the constants, three times the "helper variables" to hold results of calculations etc..

And since most filter writers are now going to feel compelled to produce both YV12 and YUY2 versions for speed reasons, that means two algorithms to maintain and bugfix.

Apart from that I love it, and can see no reason whatsoever to move to an internal 4:4:4 format :p :D.

[Marc FD]

> I disagree. This may be true if all you need to do is iterate over each
> component individually while applying some processing to them. But as
> soon as you get interdependence between the components I find a packed
> format much nicer.

no. as soon as you use complex algorithms, even with interdependance, the packing is an overhead. you need to deinterleave it.

in fact, using pseudo planar YUY2 can be faster and easier to code when you MMX optimise something more complex than cnr2 for exemple.

> It's a question of speed of execution/speed of development. YV12 is
> undeniably faster at runtime, but I find it to be at least twice as
> much typing, which means twice as many typo bugs. It also means three
> times the pointers and at least two times the constants, three times
> the "helper variables" to hold results of calculations etc..

for me, YV12 is much easier to code, and MMX optimisations are a real pleasure.
i'll port Cnr2 to YV12 today, there's interdependency in it, i'll report if it's really that hard ^^

> And since most filter writers are now going to feel compelled to
> produce both YV12 and YUY2 versions for speed reasons, that means two
> algorithms to maintain and bugfix.

YUY2->YV12 / YV12 filter / YV12->YUY2 is a very little overhead, if users don't have no YV12 input nor YV12 output.

[SansGrip]

in fact, using pseudo planar YUY2 can be faster and easier to code when you MMX optimise something more complex than cnr2 for exemple.

I have no idea which is preferable for MMX, since it's been about 10 years since I last looked at assembler ;), but my gut tells me that planar would be a lot more efficient.

for me, YV12 is much easier to code

Hmmm. Since you've written more YV12 code than me I'll take your word for it. Perhaps when I'm more au fait with planar I'll agree with you ;).

YUY2->YV12 / YV12 filter / YV12->YUY2 is a very little overhead, if users don't have no YV12 input nor YV12 output.

I suppose, though I still think there should be One True Way (i.e. a canonical internal format, probably planar 4:4:4, either 8- or 16-bit).

[trbarry]

In much of the computer industry the cost of development and especially maintenance dominates. But there are some things that are CPU intensive enough that they are worth writing in assembler or doing other things that gain performance at the expense of complication.

I think video is still like that. This is actually is some of what attracted me to writing video programs in the first place. I'm mostly an assembler programmer and video make me feel more valuable.

Anyone can attempt to write video filters in any easier high level language with data structures optimized for human critters to understand.

But you only have to run it a couple times and you may be thinking about ordering an assembler manual and doing some strange optimizations.

I like it this way. ;)

- Tom

[SansGrip]

@Tom

At the risk of straying inexcusably far from the topic:

I'm mostly an assembler programmer and video make me feel more valuable.

Well given the popularity and utility of your various filters I'd say you're definitely valuable :).

Anyone can attempt to write video filters in any easier high level language with data structures optimized for human critters to understand.

And this is how I like to write them -- the first iteration, anyway. I like to code it out fast, testing each step until it works, then go back and tweak. I find by writing the algorithm out the first way that pops into my head, I often discover a better way of doing it afterwards. I favour "extreme programming" over "big plan up front".

I also like to use a lot of asserts etc., which really slow it down. So for me performance while debugging/pre-beta is secondary to maintainable, comprehensible code. Planar YUV (at least the "standard" multi-pointer implementation of it, as opposed to some kind of efficient class library such as I'm pondering now, any help appreciated :D) and multiple colourspace support are the antithesis of these two goals ;).

Edit: As you are doubtless aware, "comprehensible" and "fast" versions are often maintained side-by-side in critical areas of commercial products, such as a spreadsheet's calculation engine. In debug builds both are run on the same data and the output compared, which is an extremely effective way of tracking down bugs in the "fast" version. This is roughly what I hope to implement into my filters when I work out a way to do it with as few keystrokes as possible ;).

But you only have to run it a couple times and you may be thinking about ordering an assembler manual and doing some strange optimizations.

I'm already thinking that :). I've been meaning to learn i386 asm (I knew 680x0 way back when) for a very long time, and this gives me a good incentive.

I like it this way. ;)

I wouldn't still be here if I didn't ;). Anyway, back to mpeg2dec...