idea for upsampling interlaced YV12 in a progressive way....

[scharfis_brain]

as we know, interlaced yv12 has this structure:

. marks luma samples of the 1st field
X marks a chroma sample of the 1st field
, marks luma samples of the 2nd field
Y marks a chroma sample of the 2nd field

1st field 2nd Field

. . . .
X X , , , ,
. . . . Y Y
, , , ,
. . . .
X X , , , ,
. . . . Y Y
, , , ,


this means, if we merge together both fields, we will get this:

line 0 . . . .
line 1 ,X, ,X, <- correct spatial position
line 2 .Y. .Y. <- correct spatial position
line 3 , , , ,
line 4 . . . .
line 5 ,X, ,X,
line 6 .Y. .Y.
line 7 , , , ,

the missing chroma of line 3 & 4 can be interpolated using :

for line 3: 66% * Y2 + 33% * X5
for line 4: 33% * Y2 + 66% * X5


this kind of yv12 upsampling will completely destroy a true interlaced video,
but may help upsampling progressive content, that has been subsampled
with interlaced YV12.



Does this make sense, or am I thinking in a totally wrong way?

[kassandro]

Originally posted by scharfis_brain

as we know, interlaced yv12 has this structure:

. marks luma samples of the 1st field
X marks a chroma sample of the 1st field
, marks luma samples of the 2nd field
Y marks a chroma sample of the 2nd field

1st field 2nd Field

. . . .
X X , , , ,
. . . . Y Y
, , , ,
. . . .
X X , , , ,
. . . . Y Y
, , , ,


I agree. A very nice visualisation, congratulation. For me interlaced yv12 is just ordinary yv12 but for each field separately.


this means, if we merge together both fields, we will get this:

line 0 . . . .
line 1 ,X, ,X, <- correct spatial position
line 2 .Y. .Y. <- correct spatial position
line 3 , , , ,
line 4 . . . .
line 5 ,X, ,X,
line 6 .Y. .Y.
line 7 , , , ,

I still agree with the visualisation, but not with the correctness assertions. The chroma in line 1 is not correct. According to the mpeg2 specification the chroma must be exactly between line 0 and line 1 and according to the mpeg1 specification it must be on line 0. By a similar reasoning also the chroma on line 2 is incorrect.
I came to the conclusion, that the best thing is simply to pretend that interlaced yv12 is identical with ordinary yv12. I strongly oppose the suggestion to use
ConvertToYUY2(interlaced=true).ConvertToYV12.

[scharfis_brain]

okay, lets forget about the vertical spatial positions...

but IMO this method should give some kind of improvement...

yv12interlaced to yv12 progressive is really bad.

look at this:

http://home.arcor.de/scharfis_brain/coloredfishes.png

[kassandro]

I don't know, what you want to say with this picture. Clearly, the two bottom pictures have some color bleeding, but also the top right picture has a little bit of color bleeding. I also do not know, how you did make them.

[tritical]

Although I get horribly confused when thinking about interlaced YV12, I do think your suggested method would (might) give better results. From my current understanding of interlaced YV12 (which may be completely wrong) this is an accurate representation of an interlaced YV12 frame:


0t
C0 = 0.75*0t + 0.25*2t
1t

2t
C1 = 0.75*3t + 0.25*1t
3t

4t
C2 = 0.75*4t + 0.25*6t
5t

6t
C3 = 0.75*7t + 0.25*5t
7t

8t
C4 = 0.75*8t + 0.25*10t
9t

10t
C5 = 0.75*11t + 0.25*9t
11t


where 1t, 2t, 3t, etc... represent the luma lines, and C1, C2, C3, etc.. represent the chroma lines. Now if the frame is truely interlaced and the two fields don't represent the same picture then to interpolate the color for line 5t when up converting to YUY2 you have to use the nearest two color lines from the same field, which in this example would be C1 and C3. However, if you've matched the fields so that they now are both from the same progressive frame, then you can use the nearest two color samples to interpolate line 5t which would be C2 and C3 using a 2/3,1/3 weighted average as scharfis suggests.

The only thing I'm worried about is that my diagram is completely and utterly wrong. Since for true interlaced upsampling to YUY2 it would suggest 3/8,5/8 weights of C1 and C3 and AviSynth currently uses 1/4,3/4 weights? Like I said, interlaced YV12 confuses me horribly.

EDITED: Actually the weights for the C2 and C3 average for line 5t considering both fields are now from the same frame would be 3/4,1/4. Which are exactly the weights for progressive YV12->YUY2 upsampling, so the whole agrument is just circular. If you match the fields to make the frame progressive again then progressive upsampling will give better results then using interlaced upconversion even though the original material was stored as interlaced. You obviously wont get the same result though as if the original material had been downsampled to YV12 using progressive downsampling.

[MfA]

In theory the vertical offsets of chroma samples alternates between 1/4 and 3/4 pel for MPEG-2/4, but not MPEG-1 if my memory serves me correct, this is expressly done so that if you just weave two fields they will end up at the correct position.

So all this isnt necessary, in theory :)

The conversion routines you used for the colored fish example are probably buggered though.

[kassandro]

Sorry, in my first contribution, I was concerned with converting interlaced yv12 to ordinary yv12. I think that is really the most important problem. Should one change anything or not? As far as upsampling from interlaced yv12 to yuy2 is concerned, scharfi is absolutely correct: the chroma lines are not between luma lines as I claimed, rather for each luma line there is a chrom line.

[MfA]

As I said, for yv12 if the thing was coded properly you can simply weave the fields and all the chroma samples will end up at the correct position ... no interpolation needed.

[tritical]

Nice how it works out :D.

Not sure if this thread is the right place to post this, but I was looking through some of AviSynth's YV12 conversion routines and in all the places a 1/4,3/4 average is performed the following code is used:


pavgb mm4,mm2 // interpolate chroma U (25/75)
pavgb mm5,mm3 // interpolate chroma V (25/75)
psubusb mm4, mm6 // Better rounding (thanks trbarry!)
psubusb mm5, mm6
pavgb mm4,mm2 // interpolate chroma U
pavgb mm5,mm3 // interpolate chroma V


I might be crazy, but for accurate rounding when doing 1/4,3/4 averages with pavgb shouldn't the psubusb instruction come before the first pavgb instruction? To make sure I wasn't crazy I worked it out and am still getting that it should.

pavgb mm4,mm2
psubusb mm4,mm6
pavgb mm4,mm2
((((a+b+1)>>1)-1) + b + 1) >> 1 = x
((a+b+1)>>1)-1 + b + 1 = 2x
a+b+1 - 2 + 2b + 2 = 4x
a+b+b+b+1 = 4x - rounds down, Error = -0.25

psubusb mm4,mm6
pavgb mm4,mm2
pavgb mm4,mm2
(((a-1)+b+1)>>1) + b + 1) >> 1 = x
((a-1)+b+1)>>1) + b + 1 = 2x
(a-1)+b+1 + 2b + 2 = 4x
a+b+b+b+2 = 4x - correct rounding, Error = 0

pavgb mm4,mm2
pavgb mm4,mm2
(((a+b+1)>>1) + b + 1) >> 1 = x
(a+b+1)>>1 + b + 1 = 2x
a+b++b+b+3 = 4x - rounds up, Error = +0.25

Or just consider the case of doing a 1/4,3/4 average of the pixel combo

10 and 20 = 17.50 = 18 (what it should be rounded to)

using pavgb, psubusb, pavgb you get 17 instead of 18. If you use psubusb, pavgb, pavgb you get the correct answer of 18. I know its just nitpicking and doesn't really make much difference, but I was just wondering.

[scharfis_brain]

hm.. very confusing here...

are we still talking about the same?

progressive film being subsampled with interlaced yv12.
I want to upsample this to yuy2 using my method above.

some crappy tests showed, that my upsample is able to nearly reach the quality as if the film had been sampled as progressive yv12

it looks much better than avisynth internal function for yv12interlaced to yuy2.

I will post the script later.
(that's why I asked for aritary weave / separatefields in the other thread. Its a pain selecting a couple of lines out of a picture within avisynth)

[MfA]

If progressive yv12 frames are stored in an interlaced stream they can be exactly reconstructed ... so why not just reconstruct them and use the progressive yv12->yuy2 conversion?

Your method looks better not because it is correct ... it looks better because you use chroma samples from 2 fields (with the wrong weights though, you should simply use 75/25). Whereas avisynths function only works with 1 field (as it should, it is interlaced for a reason). For true interlaced content your method will fuck up for obvious reasons, and telecined stuff for that matter, so I really dont see the point ... even for mixed progressive/interlaced content it makes no sense.

Hell, who wants to convert to yuy2 in the first place??? What is the point?

[scharfis_brain]

huh?

progressive YV12 has the chrom placed on other positions than interlaced YV12.


progressive YV12

. .
X <- chroma sample
. .
<- missing sample
. .
X <- chroma sample
. .
<- missing sample


interlaced YV12

. . <- missing sample

. X . <- chroma sample

. Y . <- chroma sample

. . <- missing sample



please do NOT mention the maybe wrong quarter positioning of the chroma samples.

what I want to say is:

for progressive YV12, we have a homogenus chroma information
every 2nd pixel has one chroma sample.


but for interlaced YV12, we have two following chroma samples with a distance of one active luma line, and then we have a gap of 2 missing chroma samples.

you may want to visualisze this using the UtoY() command.
just upsampling it using converttoyv12() will make it look worse, cause the chroma gets placed wrong.

[kassandro]

Originally posted by tritical
Not sure if this thread is the right place to post this, but I was looking through some of AviSynth's YV12 conversion routines and in all the places a 1/4,3/4 average is performed the following code is used:


pavgb mm4,mm2 // interpolate chroma U (25/75)
pavgb mm5,mm3 // interpolate chroma V (25/75)
psubusb mm4, mm6 // Better rounding (thanks trbarry!)
psubusb mm5, mm6
pavgb mm4,mm2 // interpolate chroma U
pavgb mm5,mm3 // interpolate chroma V


I might be crazy, but for accurate rounding when doing 1/4,3/4 averages with pavgb shouldn't the psubusb instruction come before the first pavgb instruction?


No the above code is correct! Firstly, you should mention that all the bytes of register mm6 are filled with 1. Then you should recall that pavgb makes the upward rounded averages (a + b + 1)/2. If you substract before, then you simply get the downward rounded averages (a + b)/2. While (a + b + 1)/2 has upward bias, (a + b)/2 has a downward bias. Thus if you iterate averages, one average should have upward and the other downward bias or reversely. Thus you can do both: subtracting before or after. Statistically both are equally good, though both may clearly yield different results.

[scharfis_brain]

okay, lets look at the chroma samples:

the script I have used:


avisource("colfish.avi")
a=converttoyv12().vtoy().subtitle("progressive YV12")
b=converttoyv12(interlaced=true).vtoy().subtitle("interlaced YV12")
c=converttoyuy2().vtoy().converttoyv12().subtitle("YUY2")
x=stackvertical(a,b)
stackhorizontal(c,x)


the resulting image:

http://home.arcor.de/scharfis_brain/chromaV.png

I've tried edge directed interpolation of the missing two chroma samples, and the results are looking really fine....
but that script is total crap, because of avisynths lacking abilities stacking multiple lines together

the script:

loadplugin("c:\x\tomsmocomp.dll")
avisource("colfish.avi")
l=last.converttoyuy2()
converttoyv12(interlaced=true)
stackhorizontal(utoy(),vtoy()).converttoyuy2()
tomsmocomp(-1,-1,0)
crop(0,2,0,0).addborders(0,0,0,2)
pointresize(width,height*3/4)
lanczos4resize(width,288)
crop(0,0,0,-3).addborders(0,3,0,0)
u=crop(0,0,width/2,0)
v=crop(width/2,0,0,0)
ytouv(u,v)
mergeluma(l)
converttorgb24()
subtitle("interlaced YV12 to YUY2 by scharfi")

http://home.arcor.de/scharfis_brain/yv12toyuy2-improved.png

(please compare with the interlaced yv12 of the image, I posted earlier above)

[kassandro]

Originally posted by scharfis_brain
huh?

progressive YV12 has the chrom placed on other positions than interlaced YV12.


progressive YV12

. .
X <- chroma sample
. .
<- missing sample
. .
X <- chroma sample
. .
<- missing sample


interlaced YV12

. . <- missing sample

. X . <- chroma sample

. Y . <- chroma sample

. . <- missing sample



please do NOT mention the maybe wrong quarter positioning of the chroma samples.

what I want to say is:

for progressive YV12, we have a homogenus chroma information
every 2nd pixel has one chroma sample.


but for interlaced YV12, we have two following chroma samples with a distance of one active luma line, and then we have a gap of 2 missing chroma samples.

you may want to visualisze this using the UtoY() command.
just upsampling it using converttoyv12() will make it look worse, cause the chroma gets placed wrong.
Unfortunately you are absolutely correct here, but what should we do? I decided for myself to do nothing, if a progressive film is encoded with interlaced yv12, which is an unfortunate standard on satellite tv. This is certainly the fasted solution, but doing some chroma averaging is a little bit like blended frames, which I really hate. The chroma is already quite thin in yv12 and further blurring cannot be good. That is also the key reason, why I don't like the builtin color space conversion filters. ConvertToYUY2.ConvertToYV12 shouldn't touch the yv12 chroma, but in reality it blurs the chroma.

[kassandro]

Originally posted by scharfis_brain

avisource("colfish.avi")
a=converttoyv12().vtoy().subtitle("progressive YV12")
b=converttoyv12(interlaced=true).vtoy().subtitle("interlaced YV12")
c=converttoyuy2().vtoy().converttoyv12().subtitle("YUY2")
x=stackvertical(a,b)
stackhorizontal(c,x)


You shouldn't do it this way. Rather you should split the fields of the input avi, then convert the fields separately to yv12 and weave them together again. That would be true interlaced yv12.


but that script is total crap, because of avisynths lacking abilities stacking multiple lines together

OK, I will make a multiframe weave tonight - hopefully.

[scharfis_brain]

Rather you should split the fields of the input avi, then convert the fields separately to yv12 and weave them together again. they say we should NOT use this way, cause it gives us the wrong chroma sampling.

btw. I've made my script a liitle bit more structurized:


loadplugin("c:\x\tomsmocomp.dll")

function sb_yv12toyuy2(Clip i)
{ i
stackhorizontal(utoy(),vtoy()).converttoyuy2()
tomsmocomp(-1,-1,0)
crop(0,2,0,0).addborders(0,0,0,2)
pointresize(width,height*3/4)
lanczos4resize(width,height*4/3)
crop(0,0,0,-3).addborders(0,3,0,0)
u=crop(0,0,width/2,0)
v=crop(width/2,0,0,0)
ytouv(u,v)
mergeluma(i.converttoyuy2())
}


avisource("colorspace.avi")
crop(0,0,400,300)

l=last.converttoyuy2()
m=l.converttoyv12(interlaced=true)

a=m.converttoyuy2(interlaced=true).subtitle("standard upsampling")
b=m.converttoyuy2().subtitle("MfA's suggestion")
c=l.subtitle("pure yuy2")
d=m.sb_yv12toyuy2().subtitle("interlaced YV12 to YUY2 by scharfi")


stackvertical(stackhorizontal(a,b),stackhorizontal(c,d))


the output looks like this:

http://home.arcor.de/scharfis_brain/yv12-compare.png

[MfA]

Originally posted by scharfis_brain
please do NOT mention the maybe wrong quarter positioning of the chroma samples.

Why not? I dont know who this "they" is you are repeating ... but I can read.

In each field of an interlaced frame, the chrominance samples do not lie (vertically) mid way between the luminance samples of the field, this is so that the spatial location of the chrominance samples in the frame is the same whether the frame is represented as a single frame-picture or two field-pictures.

Guess where that is from ...

Oh BTW my suggestion was not to convert the fields as if they were progressive frames ... my suggestion was to reconstruct the progressive frames first, by simple weave, and then do the conversion.

[scharfis_brain]

Originally posted by MfA
Why not? Is the truth too hard to argue with?
no, it is just, because I am currently ignoring possible half/quarter pel chromashifts, to make understanding a little bit easier.

I am talking about it in a simplified way, but no matter, exept the small chromashift, my assumption seem to be correct.



Oh BTW my suggestion was not to convert the fields as if they were progressive frames ... my suggestion was to reconstruct the progressive frames first, and then do the conversion.

huh? I cannot follow...
can you describe this a little bit more?

[MfA]

In principle if you want to store a progressive sequence in MPEG-2/4 as fields all you should do is seperatefields ... chroma should be plain subsampled and nothing more, just like luma.

That is the right way of doing it.

So if video is coded in the right way all you have to do is weave both the luma and chroma planes together to reconstruct an original progressive frame (regardless of the color space!). After that color space conversion is trivial.

[scharfis_brain]

still not understand...

if you open an mpeg-2 with interlaced yv12 chrom in avisynth, you'll get an already weaved image.

starting from this base, how would you convert this interlaced yv12 to yuy2 or progressive yv12 using a better approach than converttoxxx(interlaced=true) ?

I still do not understand, why chroma-format shouldn't matter here ?!?

we have half vertical precision in yv12.. how should that go to fullsize just by weaving to clips ???

[MfA]

If that is the case why are you doing "m=l.converttoyv12(interlaced=true)"?

That is meant to be used on fields.

[scharfis_brain]

it is meant to be used on weaved interlaced footage.
not on fieldseparated interlaced footage.

so this is correct.

[MfA]

It is not meant to be used on progressive frames though.

I should add that that has nothing to do with the spatial location of the chroma samples, but is simply because it will only use half the chroma samples for either "field" ... it just treats the frame as two fields.

Personally I think the interlaced=true flag is a huge mistake, they should have simply done a detection of the fieldbased flag in the conversion routine ... let people process fields as fields, makes much more sense. In fact this can be seen as a workaround, while the real bug (the conversion routine not respecting the fieldbased flag) is still there.

[sh0dan]

Interesting discussion. Sorry I can't spare more time for it.

Anyway - Have you tried (for moving video) to create a combmask (using fielddeinteralce or similar), and use it for separating chroma components.

Something like:

combmask = fielddeinterlace(map=true, chroma=true)
# Nasty way of converting combmask to black/white
combmask=combmask.utoy().coloryuv(cont_y=2000, off_y=-400).pointresize(width, height)
progressive = converttoyuy2()
interlaced = converttoyuy2(interlaced=true)
overlay(progressive, interlaced, mask=combmask)


Sorry for the nasty combmask conversion ;)

[tritical]

Thus if you iterate averages, one average should have upward and the other downward bias or reversely. Thus you can do both: subtracting before or after. Statistically both are equally good, though both may clearly yield different results.

Guess they are both statistically accurate, or have the same amount of relative error. If you subtract after the first pavgb then 0.25 and 0.50 get rounded down while only 0.75 gets rounded up. If you subtract before the first pavgb then 0.25 gets rounded down while 0.50 and 0.75 get rounded up. Both are equally good, I am just always use to rounding 0.50 upwards, so nm.

[Wilbert]

It is not meant to be used on progressive frames though.
True.


Personally I think the interlaced=true flag is a huge mistake, they should have simply done a detection of the fieldbased flag in the conversion routine ...
You mean interlace flag in dvd2avi? Donald was implementing auto upsampling in dvd2avi/mpeg2dec3 (on basis of the interlace flag in dvd2avi). He got stuck somewhere, so maybe you guys could help him out:

http://forum.doom9.org/showthread.php?s=&postid=506091#506091

An other problem is that this interlace flag is (apperently) a bit unreliable.

About this interlaced YV12 to progressive YV12:

Unfortunately you are absolutely correct here, but what should we do? I decided for myself to do nothing, if a progressive film is encoded with interlaced yv12, which is an unfortunate standard on satellite tv. This is certainly the fasted solution, but doing some chroma averaging is a little bit like blended frames, which I really hate. The chroma is already quite thin in yv12 and further blurring cannot be good. That is also the key reason, why I don't like the builtin color space conversion filters. ConvertToYUY2.ConvertToYV12 shouldn't touch the yv12 chroma, but in reality it blurs the chroma.

Is it possible in theory (when having the right filters) to do a interlaced YV12 to progressive YV12 in a good way?

I'm still confused about a lot of things. If I understand it correct three situations can occur (source = dvd):

1) encoded as progressive
2) progressive stuff encoded as interlaced
3) interlaced encoded as interlaced

Suppose the interlace flag from dvd2avi is true (for some frame), does that imply we have a frame where (2) or (3) is possible, or is that only (3)?

[Kika]

does that imply we have a frame where (2) or (3) is possible, or is that only (3)?

2 and 3!

[scharfis_brain]

@sh0dan:
perheps, you didn't got it...

I am talking about pure progressive Film (25p) squeezed into interlaced yv12 (as it seems to be standard on DVB and some DVDs).

so no need for combmasks here.

both, converttoyuy2() and converttoyuy2(interlaced=true) will result in a horrible chroma-blurred frame. (as showed on the quad-fish-frame)

only the function I posted some post above is able to reproduce the chroma much better.( as showed on the fish-frame, I posted later)

but this function is really shitty, because I cannot restore the correct spatial postion of the chroma due to avisynth limitations.

wilbert:
I think, that the progressive frame flag is true, for the cases (2) & (3). the frame flag doesn't care about image contents. It only cares about encoding scheme..


so, what we need for YV12 to YUY2 conversion is:

cases:
(1) converttoyuy2()
(2) sb_yv12toyuy2()
(3) converttoyuy2(interlaced=true)

or, if we have film & video mixed, we need a (chroma) combmask, which is steering, whether sb_yv12toyuy2() is used (progressive) or converttoyv12(interlaced=true) is used....

[MfA]

Originally posted by scharfis_brain
@sh0dan:
perheps, you didn't got it...

I am talking about pure progressive Film (25p) squeezed into interlaced yv12 (as it seems to be standard on DVB and some DVDs).

Well no, you are talking about progressive film squeezed into interlaced yv12 wrongly (dunno if doing it wrong is really the standard).

If they did it correctly it would NEVER be equivalent to


avisource("colorspace.avi")
crop(0,0,400,300)

l=last.converttoyuy2()
m=l.converttoyv12(interlaced=true)


It would simply use the trivial progressive convertion


avisource("colorspace.avi")
crop(0,0,400,300)

l=last.converttoyuy2()
m=l.converttoyv12


In principle the only difference between a progressive frame stored as a frame or fields in the bitstream is the storage ... all the luma and chroma samples are exactly the same, since they are in the exact same location (as that extract from the MPEG-2 standard said). Since the correct convertion for progressive frames is with interlaced=false, that is what should be used.

[tritical]

Hm, how exactly is a YUY2(interlaced)->YV12(interlaced) conversion suppose to sample? If you separate fields, then convert each field separately using normal YUY2->YV12 conversion and weave them back together then you end up with:

0t
C0 = 0.5*0t + 0.5*2t
1t

2t
C1 = 0.5*1t + 0.5*3t
3t

Which I use to think was the way to do it, but AviSynth's internal converttoyv12(interlaced=true) doesn't use the above sampling, and that isn't the way my first diagram shows it. I made a test filter with the conversion based off the sampling given in my first diagram which gives this:

0t
C0 = 0.75*0t + 0.25*2t
1t

2t
C1 = 0.75*3t + 0.25*1t
3t

but after I implemented the above conversion routine I found that it isn't the same as AviSynth's internal converttoyv12(interlaced=true) either, which actually samples like:

0t
C0 = 0.75*2t + 0.25*0t
1t

2t
C1 = 0.75*3t + 0.25*1t
3t

Which one of these three is correct? Or will they all work?

[scharfis_brain]

in fact, the sometimes (or often) don't do that correct.
the correct way to do so, is in fact a simple converttoyv12().

but often, we have to deal with progressive frame contents being in interlaced chromaspace, meaning converttoyv12(interlaced=true).

this is, what I am talking about.

I do not want to speculate about right or wrong conversions. I know, that converttoyv12() is the correct one for this case.

If I wouldn't have probs with progressive frames in interlaced yv12, I wouldn't have made this thread.

btw. the chroma samples for progressive & interlaced yv12 aren't in the same position, please look again at the gray-tone image with the fishes....

finally some more comparision:



loadplugin("c:\x\tomsmocomp.dll")

function sb_yv12toyuy2(Clip i)
{ i
stackhorizontal(utoy(),vtoy()).converttoyuy2()
separatefields()
e=selecteven()
o=selectodd().coloryuv(gain_u=200)
interleave(e,o).weave()
tomsmocomp(-1,-1,0)
shift=2
crop(0,shift,0,0).addborders(0,0,0,shift)
pointresize(width,height*3/4)
lanczos4resize(width,height*4/3)
crop(0,0,0,-2).addborders(0,2,0,0)
u=crop(0,0,width/2,0)
v=crop(width/2,0,0,0)
ytouv(u,v)
mergeluma(i.converttoyuy2())
}


avisource("colfish.avi")
lanczosresize(480,height) #enlaring the image horizontally for the text

a=separatefields().converttoyv12().weave().sb_yv12toyuy2().subtitle("sepfield()s.converttoyv12().weave().sb_yv12toyuy2()")
b=converttoyv12(interlaced=true).sb_yv12toyuy2().subtitle("converttoyv12(interlaced=true).sb_yv12toyuy2()")

c=separatefields().converttoyv12().weave().converttoyuy2(interlaced=true).subtitle("sepfields().converttoyv12().weave().converttoyuy2(interlaced=true)")
d=converttoyv12(interlaced=true).converttoyuy2(interlaced=true).subtitle("converttoyv12(interlaced=true).converttoyuy2(interlaced=true)")

e=separatefields().converttoyv12().weave().converttoyuy2().subtitle("sepfields().converttoyv12().weave().converttoyuy2()")
f=converttoyv12(interlaced=true).converttoyuy2().subtitle("converttoyv12(interlaced=true).converttoyuy2()")

stackvertical(stackhorizontal(a,b),stackhorizontal(c,d),stackhorizontal(e,f))

converttorgb24()

http://home.arcor.de/scharfis_brain/coloredfishes-quad.png

[MfA]

Originally posted by tritical
Hm, how exactly is a YUY2->YV12 interlaced conversion suppose to sample?

It should just do a progressive conversion on the original frames, and then seperatefields ... nothing more.

In each field of an interlaced frame, the chrominance samples do not lie (vertically) mid way between the luminance samples of the field, this is so that the spatial location of the chrominance samples in the frame is the same whether the frame is represented as a single frame-picture or two field-pictures.

Meah culpa for distracting from the conversation to this extent scharfis, but it would have been easier if at the start we could have established that what you wanted was a way to clean up bad color space conversions :) (ie. field based color space conversions on progressive content)

[Wilbert]

About this interlaced YUY2 -> interlaced YV12 conversion:

It should just do a progressive conversion on the original frames, and then seperatefields ... nothing more.
? In that case chroma from one of the original YUY2 fields bleeds into both YV12 fields.

@tritical,

According to figure 4 in

http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html

your first diagram should be correct. If Sh0dan used the second diagram, I think that's not correct.

[MfA]

So? There are no real fields, they are just an irrelevant aspect of the way the sequence is stored in the bitstream ... it is progressive content after all. The chroma samples as the luma samples from both "fields" are from the same temporal location ... so using them to compute the new chroma samples introduces no artifacts, there is no bleeding just blending.

I just noticed you might have not started from the beginning ... I know it sounds ridiculous, but all this thread just concerns progressive content stored as fields (or more precisely, wrongly stored ... otherwise you would just weave the fields and do a progressive color space conversion and not worry about it).

From the topic starter :

this kind of yv12 upsampling will completely destroy a true interlaced video, but may help upsampling progressive content, that has been subsampled with interlaced YV12.

[Wilbert]

So? There are no real fields, they are just an irrelevant aspect of the way the sequence is stored in the bitstream ... it is progressive content after all. The chroma samples as the luma samples from both "fields" are from the same temporal location ... so using them to compute the new chroma samples introduces no artifacts, there is no bleeding just blending.

I just noticed you might have not started from the beginning ... I know it sounds ridiculous, but all this thread just concerns progressive content stored as fields
Yes, yes I know. tritical started asking about YUY2(interlaced) to YV12(interlaced) conversions, not me.

[tritical]

I am also sorry for taking the thread so far off topic. However, since you are using converttoyv12(interlaced=true) in your scripts to test I think it is somewhat relevant how it is performed. Since my original suggestion was that if a frame is down-sampled to yv12 using interlaced conversion but the fields belong to the same picture you will get better results if you use the progressive yv12->yuy2 conversion when up-sampling. However that only holds if the conversions are performed how I thought they were, otherwise the 25/75 averages in the progressive conversion do not line up! Here is an example taking a progressive YUY2 frame, converting to yv12 interlaced using each of the three methods I described above, and then converting back to YUY2 using the progressive conversion. (I added the straight progressive yuy2->yv12 and back for comparison).

http://www.missouri.edu/~kes25c/png-testconvert.PNG

Now the interesting thing that I am just now understanding is that if the conversion is done via separatefields().converttoyv12().weave() then your method will give better results for doing yv12(interlaced)->yuy2 progressive then using the built in yv12->yuy2 progressive conversion!

since you would have:

0t
C0 = 0.5*0t + 0.5*2t
1t

2t
C1 = 0.5*3t + 0.5*1t
3t

4t
C2 = 0.5*4t + 0.5*6t
5t

6t
C3 = 0.5*5t + 0.5*7t
7t

which is effectively (spatial chroma positions will now be technically wrong, but this is where they should be according to the averages used for the pixels above, and this is what scharfis has been thinking all along i take it)

0t

1t C0 = 0.5*0t + 0.5*2t

2t C1 = 0.5*3t + 0.5*1t

3t

4t

5t C2 = 0.5*4t + 0.5*6t

6t C3 = 0.5*5t + 0.5*7t

7t



in which case 1t and 2t now have values w/o interpolation and to get 3t and 4t you would use 2/3,1/3 averages. Took me forever to realize the point of the first post :o.

[MfA]

Okay Wilbert, I read that question as converting a progressive yuy2 frame to two yv12 fields for an interlaced stream.

BTW with the fieldbased flag I simply meant the avisynth one, if the fieldbased flag is set it should not use the progressive colorspace conversions ... it should determine wether it was an odd/even field and use an appropriate field based colorspace conversion. As it is now you cannot work with an interlaced stream as just fields if you want to do colorspace conversion. Ironically you are forced to do frame based processing to process field based content.

[scharfis_brain]

*grrrrrrrrrrr*

It seems, that I was hunting ghosts...

After some other tests, I found out, that the interlaced chroma subsampling of TMPGenc & AVISynth differs.

avisynth's color subsampling seems to be much more jaggy, while the one tmpegenc does, was more smoothed....
(similar to the separatefields().converttoyv12().weave() method)

so, MfA, you're right. AVISynth has bugs according YV12...

I thought I could correct some videos, but their subsampling is similar to tmpegenc's method.

I never found avisynth's subsamling on my sources...

so I apoligize for any confusions.....

[kassandro]

Originally posted by scharfis_brain

avisynth's color subsampling seems to be much more jaggy, while the one tmpegenc does, was more smoothed....
(similar to the separatefields().converttoyv12().weave() method)

We should have converttoyv12(interlaced=true)=separatefields().converttoyv12().weave(), because it is the sole correct method to do it (of course internally one could avoid the weave bitblts). Now, when I inspected the avs 2.54 source, I found that there was a separate assembly routine for converttoyv12(interlaced=true), though one could have just applied the assembly routine for converttoyv12(interlaced=false) to the two fields separately. This observation doesn't necessarily imply that Avisynth's routine is incorrect. I only checked converttoyv12(interlaced=false). Similarily we should have converttoyuy2(interlaced=true)=separatefields().converttoyuy2().weave().

[sh0dan]

I just reviewed the code, and there seems to be some inconsitencies. (sp?)

Progressive YUY2 -> YV12 does a simple 50/50 downsampling. This makes sense, as 4:2:0 chroma lies directly between the 4:2:2 chroma.

Interlaced YUY2 -> YV12 does a 75/25 downsampling skipping the fields of the other field. I noticed a bug here: It seems the furthest field gets a 75% weight in the upper field, which is wrong. This should of course be corrected.

I we talk about Figure 4 (http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html), separating and converting (and doing a 50/50 downsampling) would indicate that chroma lies exactly on the line between the field luma. For example we assume that chroma lies exactly on line 2 for the pixels in line 1+3. According to fig. 4 this is not the case, and 75/25 would make more sense if fig. 4 is true.

We can agree that the current interlaced=true is incorrect - I would however like some more arguments that fig. 4 is wrong.

[scharfis_brain]

oh, that article cleared up things to me.

this means a better interlaced YV12 top YUY2 upsample should be done using a comb/motionmask for switching between progressive & interlaced upsampling, as you've told about...

what about tricticals method creating interlaced YV12 ?

[MfA]

Yes, fieldbased conversion need its own routine with even/odd detection and appropriate 75/25 mixing ... you need yuy2->yv12 conversion routines for progressive, weaved/interlaced and fieldbased (you could probably combine the last two in one routine, since the first in effect just does the last for two fields).

Cleaning up progressive video for which the color conversion was done wrong belongs in a plugin.

[tritical]

My method of creating interlaced YV12 was just what shodan was suggesting... the 75/25 method. The only problem with AviSynth's internal converttoyv12(interlaced=true), and the only difference to mine, was that it used 25/75 on the top field where it should use 75/25 as shodan pointed out. After reading that article and the mpeg2 spec again I believe this is the correct way. First, that it is the only way the averages correctly line up if you match the fields and use progressive upsampling on video that has been stored as interlaced, and secondly, because of this quote that MfA has shown before:

In each field of an interlaced frame, the chrominance samples do not lie (vertically) mid way between the luminance samples of the field, this is so that the spatial location of the chrominance samples in the frame is the same whether the frame is represented as a single frame-picture or two field-pictures.

I don't believe separatefields().converttoyv12().weave() is a technically accurate approach, however that doesn't mean it might not be used in other places/programs.

[kassandro]

Originally posted by sh0dan

I we talk about Figure 4 (http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html), separating and converting (and doing a 50/50 downsampling) would indicate that chroma lies exactly on the line between the field luma. For example we assume that chroma lies exactly on line 2 for the pixels in line 1+3. According to fig. 4 this is not the case, and 75/25 would make more sense if fig. 4 is true.

We can agree that the current interlaced=true is incorrect - I would however like some more arguments that fig. 4 is wrong.

fig. 4 may be the official specification, but certainly not the practical standard. If correct, encoding progressive content as interlaced would be better, but encoding interlaced as interlaced would be worse, because it treats a line pair assymetrically. fig.4 violate the general principle, that an interlaced frame consists of two fields, which are itself like progressive frames with half the height. It would force encoder and decoder designers to develop different routines for progressive frames and fields of interlaced frames. In practice they certainly will encode and decode a field of an interlaced frame as a progressive frame with half the height (xvid developers probably can decide this question).
Moreover, if fig. 4 would be correct, then one would have to use the weight 0.33, 0.66 instead of 0.25, 0.75 for downsampling. Of course the weight 0.33, 0.66 is not practical because a division by 3 is fairly slow in integer arithmetic.
Now the authors of the article prefer propaganda over correctness. When they illustrate the chroma upsampling bug, i.e. upsampling a progressive frame as an interlaced frame, then they use simple doubling of chroma lines for upsampling and then the situation becomes really messy if a progressive frame is upsampled as an interlaced frame. However, if upsampling according to fig.3/4 would be used the difference would be much smaller.
Finally, Microsoft and probably graphics hardware designers as well don't follow even the official specs for progressive upsampling. They only interpolate the odd lines, which is the specification for mpeg1-yv12. Doing it this way they get the nice property that converttoyuy2.converttoyv12 leaves the frame unchanged. Similarily when upsampling from yuy2 to yuv 4:4:4 only the chroma of pixels with odd x coordinate is interpolated. The official specs are simply not well thought enough to be strictly implemented in practice. Especially fig. 4 would simply be an intellectual disaster.

Sorry, the weight 0.25, 0.75 was correct, not 0.33, 0.66!

[tritical]

I took a look at mpeg2dec's yv12->yuy2 upsampling conversions just to get an idea of what downsamplings they would imply, and to see how someone else thought they should work. For progressive it uses 75/25 averages, just like the AviSynth routine and as would be suggested by straight 50/50 downsampling. Of course no one is arguing about the correctness of this one :D. For its yv12->yuy2 interlaced upsample it uses 3/8,5/8 and 1/8,7/8 averages, which are exactly the averages suggested if an alternating 75/25 yuy2->yv12 interlaced downsampling is used.

[IanB]

Browsing thru the RFE's I found a request for direct convertions between YV12 interlaced and YV12 progresive. An example was given of doing it the current slow way via YUY2. i.e.Mpeg2source("interlaced source")
ConvertToYUY2(interlaced=True)
ConvertToYV12(interlaced=False)
# Progressive style filter chain

or

Mpeg2source("progressive source")
ConvertToYUY2(interlaced=False)
ConvertToYV12(interlaced=True)
# Interlaced style filter chainMy first reaction was doing the 2 conversions was a bit mad, but his point was some filters were only YV12 and assumed either interlaced or progresive layout. After a bit, it struck me that such functions are the same in both directions and are in effect a [1 2 1] vertical blur of the chroma.

Have I got this correct? Is there any method to do this without halving the vertical chroma resolution?

I lashed up the following test script to exercise the issue and initially the results were pretty much as I expected. After looking at it for a while I am not so sure this is what should be happening.

The script generates alternating groups of lines of opposite colours with increasing pitch (1, 2, 4, 8 & 16). The left of each group is the original, the right has been converted to YV12 and back to YUY2 with all the (mis)combination of interlace=true/false. The top group is for reference and is a simple vertical chroma blur. You will need the latest 2.5.5 CVS (prior to this the point resizer was 1 pixel off and the interlaced YV12 conversion was wrong, the script show both problems nicely) Note the output is point resized x4 to allow easy viewing, you don't normally see this much detail.

For pitches above 4 pixels I am thinking there should be no lose of resolution, but there is marked chroma bluring. The effect is somewhat pathelogical as it does not appear the same for each colour pair (the yellow/blue is hardly noticable).

Thought provoking test pattern scriptGlobal H=16
Global W=20

Function MakeClip(String Pixel) {
Grey = BlankClip(100, W, H, pixel_type=Pixel, color=$3f3f3f)
Blue = BlankClip(100, W, H, pixel_type=Pixel, color=$3f3fcf)
Green = BlankClip(100, W, H, pixel_type=Pixel, color=$3fcf3f)
Cyan = BlankClip(100, W, H, pixel_type=Pixel, color=$3fcfcf)
Red = BlankClip(100, W, H, pixel_type=Pixel, color=$cf3f3f)
Magenta= BlankClip(100, W, H, pixel_type=Pixel, color=$cf3fcf)
Yellow = BlankClip(100, W, H, pixel_type=Pixel, color=$cfcf3f)
White = BlankClip(100, W, H, pixel_type=Pixel, color=$cfcfcf)
Interleave(Red, Green, Green, Red,
\ Magenta, Cyan, Cyan, Magenta,
\ Yellow, Blue, Blue, Yellow,
\ white, Grey, Grey, White)
AssumeFieldBased()
Weave()
}

Function Tests(Clip C1, Clip C2, Clip C3, Clip C4, Clip C5, Bool B1, Bool B2)
{
StackHorizontal(
\ C1, C1.ConvertToYV12(interlaced=B1).ConvertToYUY2(interlaced=B2),
\ C2, C2.ConvertToYV12(interlaced=B1).ConvertToYUY2(interlaced=B2),
\ C3, C3.ConvertToYV12(interlaced=B1).ConvertToYUY2(interlaced=B2),
\ C4, C4.ConvertToYV12(interlaced=B1).ConvertToYUY2(interlaced=B2),
\ C5, C5.ConvertToYV12(interlaced=B1).ConvertToYUY2(interlaced=B2))
PointResize(Width()*4, Height()*4)
AddBorders(0, 1, 0, 1, Color=$000000)
AddBorders(0, 2, 0, 0, Color=$FFFFFF)
}

Function BlurT(Clip C1, Clip C2, Clip C3, Clip C4, Clip C5)
{
StackHorizontal(
\ C1, MergeChroma(C1, C1.Blur(0, 1)),
\ C2, MergeChroma(C2, C2.Blur(0, 1)),
\ C3, MergeChroma(C3, C3.Blur(0, 1)),
\ C4, MergeChroma(C4, C4.Blur(0, 1)),
\ C5, MergeChroma(C5, C5.Blur(0, 1)))
PointResize(Width()*4, Height()*4)
AddBorders(0, 1, 0, 1, Color=$000000)
SubTitle(" 1")
SubTitle(" 2",W* 8)
SubTitle(" 4",W*16)
SubTitle(" 8",W*24)
SubTitle("16",W*32)
}

MakeClip("YUY2")
V1 =Last
V2 =PointResize(W, H*2, 0, 0, W, H)
V4 =PointResize(W, H*2, 0, 0, W, H/2)
V8 =PointResize(W, H*2, 0, 0, W, H/4)
V16=PointResize(W, H*2, 0, 0, W, H/8)

StackVertical(
\ BlurT(V1, V2, V4, V8, V16).SubTitle("Chroma Blur", 8, 40),
\ Tests(V1, V2, V4, V8, V16, false, false).SubTitle("False False"),
\ Tests(V1, V2, V4, V8, V16, false, true ).SubTitle("False True"),
\ Tests(V1, V2, V4, V8, V16, true, false).SubTitle("True False"),
\ Tests(V1, V2, V4, V8, V16, true, true ).SubTitle("True True") )
IanB

[scharfis_brain]

like MfA stated, interlaced YV12 and progressive YV12 share the SAME Chroma positions.

this means: if you have progressive video sampled to interlaced YV12, then just treat the interlaced YV12 is if it would be progressive. THis should retain the most crispness.

to understand, what I mean you may assume this NOP :
converttoyuy2(interlaced=false).converttoyv12(interlaced=false)

for progressive YV12 to interlaced YV12 it is the same thing, only vice versa.

MfA: Am I right here?

[MfA]

I dont agree with the term "interlaced yv12" when talking about progressive frames, that legitimizes what in essence is a badly fucked up way of treating video.

If you store progressive video in fields in MPEG-2/4 what you SHOULD do, is do all processing including colorspace conversion on frames and just do seperate fields as the final operation before encoding. This way when the video ends up in framebased mode in avisynth it is simply progressive video again. So yes you are right in that case.

What I dont know is if whether progressive video regularely is treated wrongly, with the the colorspace conversion being done on fields (that is what interlaced=true means, the fields are handled seperately). If that has occurred treating the frames as regular progressive frames is almost certainly still better than doing the yv12->yuy2 conversion seperately on fields (aka interlaced=true). So in principle you are still right, but there might be ways of cleaning up the artifacts this poor handling of the video before encoding caused though.

That would be something for a plugin though.

Marco

PS. I would like to reiterate that the fact that yuy2<->yv12 use the standard colorspace conversion routines when working in fieldbased mode should be considered a bug :) If the bug is going to remain in place I think it should be mentioned in the docs, for those few sane people who do like to process interlaced video fieldbased.

[IanB]

So we agree YV12i <-> YV12p would result in a [1 2 1] vertical chroma blur (and is a really bad idea) :p

@MarcoOriginally posted by MfA
PS. I would like to reiterate that the fact that yuy2<->yv12 use the standard colorspace conversion routines when working in fieldbased mode should be considered a bug :) If the bug is going to remain in place I think it should be mentioned in the docs, for those few sane people who do like to process interlaced video fieldbased. Would you like to propose some "warning text" to put in the doco.:D

IanB