Color problem serving decoded AVIs

[Cela]

Hi laserfan,

maybe you experienced forum guru can give my some advice on another, slightly off-topic matter, a color and avs problem, which worries me for a long time. I did a lot of reading and tests, but invain. I would like to understand what there really happens:

I have AVIs of this type:
Video
Format : MPEG-4 Visual
Format profile : Streaming Video@L1
Format settings, BVOP : No
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 2mn 14s
Bit rate : 4 681 Kbps
Width : 852 pixels
Height : 576 pixels
Display aspect ratio : 1.479 // should finally become 16/9 by transcoding by CCEB
Frame rate : 25.000 fps
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.382
Stream size : 75.0 MiB (75%)
Writing library : XviD 55

and get color problems illustrated in the attached screenshot assembly:
1) Why that difference in VirtualDub's i- and o-pane? (o-pane looks ok, i-pane has blocky color contour lines)
2) + 3) AviSource variants and DirectShowSource lead to different color effects

avs-Versions 1 & 2 are usable for editing in VD and produce good transcoding results with CCE Basic.

avs-Version 3 has a GREEN i-pane, which is not so good for editing, and produces false colors.

Where is my error? What is wrong with my footage and/or my avs? What should I make better?

Hope you can solve the puzzle.

Thanks!

[Cela]

Hi neuron2,

thanks for moving this post to this more appropriate location. Though laserfan may not find it here.

I am also not quite sure what option is the correct one to choose in DGIndexNV and DGIndex: Video | YUV -> RGB | PC_Scale or TV-Scale ?
I guess this choice defines some property for the output stream (since, I guess again, DGIndexNV and DGIndex will know very well the properties of the input stream by themselves).
(Please feel free to transfer this question to your forum.)

@all
Maybe my problem start right here, choosing the wrong option in DGIndex?
Or, may it origin by transcoding to Xvid? I suppose the profile is set to "RGB" at analog capturing.
(Depending of the source of the Xvid version of the footage.)

Can you, neuron2, or anybody else, please, help me understand this very basic issue?

Thanks in advance!

[Guest]

Please post a link to an unprocessed source sample.

I do have to ask where these files are coming from also.

[Cela]

Originally Posted by neuron2

Please post a link to an unprocessed source sample.

I do have to ask where these files are coming from also.

The sources are DVBViewer Pro recordings in TS mode from DVB-S and DVB-S2 Sat TV. Other sources origin from my AVCHD camcorder. In very rare occasions i get analog AVI recordings from a friend.
Only in the rare cases where the original footage needs some filter processing (i.e. DeShake, Neat Video, X-fade, etc.), I use an intermediate AVI step with VirtualDub filter processing. My concern is to make this intermediate step as 'lossless' and 'free of side-effects' as possible. (Though, I prefer to use i-frame only 10000kbps Xvid instead of 'lossless' codecs for the intermediate step because of speed and moderate file sizes involved.)

Please find detailed data and info in PM.

Thanks for your kind help!

[Cela]

No ideas?
Funny colors using AviSource. Why?
Colors OK using DirectShowSource!
Examples: screenshots below.

[Guest]

Post links to the original TS and the converted AVI for the case you showed and describe exactly how you made the AVI from it. I would guess that the TS->AVI step is the problem.

[Cela]

Originally Posted by neuron2

Post links to the original TS and the converted AVI for the case you showed and describe exactly how you made the AVI from it. I would guess that the TS->AVI step is the problem.

Unfortunately, I accidently deleted the ts of the first screenshot example. I will post a link to another case which will have the same problem. Practically all of my cases suffer the "funny color" and the "color contourline" problem!

TS->AVI: in avs of that sort (here AviSource options commented out):

# # TG7toAVI.avs
# ************
#
# Input:
# ******
strTG7video="__vid__" # <--- input actual value <--- !!!
strTG7audio="__vid__.ac3" # <--- input actual value <--- !!!
#
# TGG7 MTS 1440x1080/50i von AVCHD\BDMV\STREAM
# Output:
# AVI, ffdshow, 10000 kbps, K-frame only, 852x576/25p
# ****************************************************
#
LoadPlugin("D:\_My_Progs\DGAVCDecNV\dgdecnv2007\DGDecodeNV.dll")
LoadPlugin("D:\_My_Progs\VirtualDub\NicAudio\NicAudio_204\NicAudio.dll")
video=DGSource(strTG7video, deinterlace=1, resize_w=852, resize_h=576)
audio=NicAC3Source(strTG7audio, Channels=2, DRC=1) # downmix to 2 ch, apply normal DRC
video=video.ConvertToRGB(matrix="rec709") # for instant display in VirtualDub
AudioDub(video,audio)

and/or

# TS or AVI to AVI for VirtualDub and CCEB.avs
# *************************************
DirectShowSource("A380_SZG_Ueberflug4.TS",25,true,true,true) # ok for CCE
#AviSource("A380_SZG_Ueberflug4.TS").ColorYUV(levels="PC->TV") # ok for CCE
#AviSource("A380_SZG_Ueberflug4.TS") # wrong colors!
ConvertToYV12()
LanczosResize(720,576,0.0,0.6)

I open the avs in VirtualDub and get the color problems in the i-pane while playing. At pause or stop the video in the i-pane 'collapses' to the good picture, which is always displayed in the o-pane.

The funny color problem is also to be seen in the output files of CCEB and VirtualDub (Save as... in Full processing mode).
The contourline problem is not inherited by the output data.

[Guest]

Can't help without the source TS. Why bother posting without that?

[Cela]

Here you are:
color_problems_example.rar
http://rapidshare.com/files/41233751...ample.rar.html (~ 37 952 KB)

"workflow steps.txt" contains explanation of the workflow with step 1 to step 4, from ts to avi, and step5, containing the errors, and screenshots of the intermediate results.
The sample clip "ORF2 HD Test.ts" (17,259 KB) is included.
(Problem does not depend on the specific recorded program. Similar results with all my favorite tv stations, including "arte HD", etc.)

[Guest]

OK, thank you, investigating...

[Guest]

In VirtualDub Options/Preferences/Main/Output color depth, set to 24-bit (TrueColor). Also try "match display".

Does that get rid of the contouring?

Still investigating the weird colors. So far I cannot reproduce it. But I am not making an intermediate AVI. Can you get it when using just the TS file served with DGSource()? If so tell me exactly how, because your instructions are unclear for that. E.g., the two scripts (for AVI and TS) differ only in a comment.

In your sample package, why are you deinterlacing a progressive stream?

If you have an interlaced stream, then set deinterlace=1 and use_pf=true for DGSource().

[IanB]

Don't bang your head against a brick wall, you may damage the wall

The width of the input source file is not mod 2**n enough for something in your processing path.

... Width : 852 pixels

852 is only mod 4, many components need mod 16 width to work properly.

[mariush]

As said above, it may be a Virtualdub rendering issue. Play the AVS file in Media Player Classic Home Cinema and see if it happens there too.

I've noticed something like what you see in Virtualdub when I had some movie playing (which uses Overlay or some specific rendering method) and Virtualdub had to fall back to another method of displaying the movie, which didn't work quite right.

Even double check your display's resolution and color depth - you may have played a game before and that maybe crashed or set the display to 16 bit...

[Cela]

Thanks, for your answers.

Found an error in In 'TS using AviSource.avs', line 3: I forgot to change '.avi' to '.ts'

@neuron2
> In your sample package, why are you deinterlacing a progressive stream?
Because I found out by testing with 'deinterlace=1' and 'deinterlace=0' that this does not make any difference with my sort of footage. So I simply copied my standard avs template. If you advice me to correct this, I certainly will.

> But I am not making an intermediate AVI.
To me the 'intermediate AVI' is a final result of VirtualDub.
From what I see when I use it again, I am worried that it my have some internal defect. If so, I would like to make it better. That is the main motivation for having raised these questions.
I would expect that it should be possible to re-use output as input. Repetitive process cycles should be possible and are frequently needed.

>Still investigating the weird colors. So far I cannot reproduce it. But I am not making an intermediate AVI.
The problem starts to show itself in step 5. So it may origin in step 4 or earlier (as hidden problem).
When I play the "TS using DirectShowSource.avs" in VirtualDub it shows in the i-pane the 'contour' issue, but not the 'funny color' issue.
The screenshot "TS or AVI using DirectShowSource - color contour problem.jpg" shows my results with ".ts".
Corrected 'TS using AviSource.avs' doesn't open a TS. Error message: 'line 3 Error could't open file '...ts'

May I repeat that question from post #2:

I am also not quite sure what option is the correct one to choose in DGIndexNV and DGIndex: Video | YUV -> RGB | PC_Scale or TV-Scale ?...

DGIndexNV and DGIndex will know very well the properties of the input stream by themselves. So I guess that choice must indicate a wanted property for the output. Is that correct? But, the real media output is produced later by the sw, which uses the frame serving? So why decide here and again in CCE?
Could you give a short explanation, please.

@IanB
>852 is only mod 4, many components need mod 16 width to work properly
The input is as it is. But all avs resize, LanczosResize(720,576,0.0,0.6), already to mod 16 values. Thus VirtualDub only sees 720x576.
I doubt, cropping width to 848 would help ? I'll try it.
What, exactly, do you suggest?

@mariush
'Contour' issue is not visible in my MPC-Homecinema 1.3.1249.0 nor in Nero9 ShowTime.
Its confined to VirtualDub's i-pane and not propagated elsewhere. From that I guess I needn't bang my head against a brick wall.
Re-confirmed by tests with:
TS using DirectShowSource.avs
AVI using DirectShowSource.avs

'Funny color' is visible in my MPC-Homecinema 1.3.1249.0 and in Nero9 ShowTime and propagates to CCE. It is not confined to VirtualDub's i-pane. From that I guess, I sure need to worry.
I need VirtualDub output which I can re-use in VirtualDub and use elsewhere!
Re-confirmed by tests with:
AVI using AviSource.avs

@all
Would you agree to this summary?
a) Do not worrry about 'contour' issue. It's an i-pane display thing, in VirtualDub, only.
b) 'funny color' issue:
-> re-using a 'Full processing mode output' AVI (from VirtualDub Step 4) via
'AVI using DirectShowSource.avs' is oK!
'AVI using AviSource.avs' is NOT OK!
-> using original TS via
'TS using DirectShowSource.avs' is OK!
'TS using AviSource.avs' is not possible, produces error "couldn't open file '... .ts' in line 7..." Which means AviSource couldn't open the original ts.

-> The issue narrows to: AviSource, DirectShowSource and VirtualDub Full processing mode in step 4 (user's view from outside)
What makes the difference between AviSource and DirectShowSource in VirtualDub regarding to funny colors???
(expert's view to what's going on below the hood.)
Finally, how to get rid of 'funny colors' in VirtualDub step 4?

Looks like good progress! And that we will catch the bug in next round!

[Guest]

Originally Posted by Cela

'TS using AviSource.avs' is not possible, produces error "couldn't open file '... .ts' in line 7..." Which means AviSource couldn't open the original ts.

Duh, of course not. TS is not AVI!

TV/PC scale is irrelevant here. It affects only display in DGIndex.

I suggest that you follow IanB's advice for your "funny color" problem and report back the results. Crop to mod16 before saving your intermediate AVI.

I can't help further because I do not want to install ffdshow.

[Cela]

Originally Posted by neuron2

...I suggest that you follow IanB's advice for your "funny color" problem and report back the results....

IanB's advice did it! Thanks!

Originally Posted by neuron2

...Crop to mod16 before saving your intermediate AVI....

To avoid the the "funny color" problem it was sufficient to enter a mod16 compliant width into

video=DGSource(strTG7video, deinterlace=0, resize_w=720, resize_h=576) # output to 720x576

in workflow step 2 before saving the intermediate AVI.

So the "funny color" problem had its origin there.
Edit: The above line should read:
So the "funny color" problem could be avoided by correct usage of DGSource(), i.e. "resize_w=[B]720[/B" (see next posts!)
Please excuse my inadequate English! It's not my first language!

I also tried Huffyuv instead of Xvid. It has no effect on these problems. But picture quality does not degrade. It remains as excellent as the original, at the cost of processing time and filesize.

Problem solved!
Thank you all for your kind help!

@mod
For housekeeping, please feel free to delete all my redundant posts and screenshots in this thread.

Regards,

Cela

[Guest]

Originally Posted by Cela

So the "funny color" problem had its origin there.

Maybe the "origin" is setting a non-mod16 size in DGSource(), but the root cause, and the thing that requires fixing, is elsewhere: Xvid, ffdshow, and/or AVISource(). I don't want people to think that DGDecNV is broken when it is not.

IanB is more qualified to speak about where the true bug actually is.

[Cela]

Originally Posted by neuron2

IanB is more qualified to speak about where the true bug actually is.

Sure, DGSource() is ok! More than that, it is the best I know! and IanB is more qualified to speak about where the true bug actually is.

In addition, DGSource() obviously has a built in power to stop propagation of certain problems induced by the footage. This is very remarkable and can be a big advantage for the user! It enables the user to make the best out of a problematic source, simply by using mod 16 compliant resize values!

Correct usage of DGCource(), resize_w=720 produced I]correct results[/I]!

But taking the input footage as ist is in my example, for which I needed help with debugging, IMHO, the bug "was not disabled" by wrong usage of DGSource().

This is what I intended to express. Obviously my English is not appropriate. Sorry!
I will edit the false wording in post #16. It should read "... could be avoided"!

Please remember, I always asked "What did I wrong?, What could I do better?" (and not "which used component is broken?").

The problem explicitly manifested itself later, after usage of LanczosResize(720,576,0.0,0.6) in the next avs played in VirtualDub.

By the way, ffdshow Xvid's documentations says, Xvid needs mod 2!

And, you are right, there is still VirtualDub in the workflow chain. This thread did, IMHO, not really clarify if my usage of VirtualDub (via avs) could do some harm to color(space) quality.

What I learned from this example:
Each workflow chain needs special care in observing all the rules for usage of each gem in the chain! Often the problems propagate and origin not where you observe the error effects.

But sure, DGSource() is the best if used correctly!
I always use it for frame-serving my h.264 footage!

Thanks for your help!

[Guest]

Thanks for the clarification (and your kind words), but I want to clarify that I wasn't criticizing you but merely clarifying.

[IanB]

In reference to the images posted in Post #5 with the skewed chroma.

What version of Avisynth.dll are you using?

AviSource() in the various versions assume different memory layout of the video frame returned by the ICDecompress() VFW library call.

I believe in version 2.5.8 it should be correct and in version 2.6 it is wrong.


For reference with YV12 decoding :-

The Y plane scan lines are padded to a multiple of 4 bytes, i.e. DWORD aligned.
The U and V planes scan lines are one half the padded width of the Y plane scan lines.


In the current 2.6 alpha all the plane scan lines are assumed padded to a multiple of 4 bytes, this is wrong.