Minimum color space conversion during vidoprocessing with Virtual Dub

[Kein]

I mainly use 2 sources for the video:

1. Direct read from videofiles recorded by FRAPS (FPS1,YUV12, rec 709, full range (https://encodingtalk.com/threads/encoding-fraps-videos-solving-colorspace-issues-wrong-color-faded-etc.1537/)) or DxTory (xtor, YUV24|CompressV3)

2. Vegas with DebugMode FrameServer, RGB24/32

In order to receive proper colorspace and color compatibility in final AVC1 MP4 video, I either use x264vfw in VirtualDub's full processing mode with the option "Convert to YUV 4:2:0" in the encoder itself (+ specify the matrix), or use "convert format" and "alias format" when it comes to External Filters.

In case with direct access to file there isn't that much to do, Virtual Dub does conversion to 32b RGB in "full processing" and then later encoder does another conversion into YUV420 again. Nothing can be done here, fine detail loss quite noticeable even with lossess codes (Lagarith, Huff, x264 lossess mode) but whatever.

My main concern is case with frameserver. Would it make more sense to send output to VirtualDub through FS in 32b RGB or the final result won't be very different? My main concern primarily is to reduce colorspace conversion as mcuh as possible.

[poisondeathray]

I mainly use 2 sources for the video:

1. Direct read from videofiles recorded by FRAPS (FPS1,YUV12, rec 709, full range (https://encodingtalk.com/threads/encoding-fraps-videos-solving-colorspace-issues-wrong-color-faded-etc.1537/)) or DxTory (xtor, YUV24|CompressV3)

2. Vegas with DebugMode FrameServer, RGB24/32

In order to receive proper colorspace and color compatibility in final AVC1 MP4 video, I either use x264vfw in VirtualDub's full processing mode with the option "Convert to YUV 4:2:0" in the encoder itself (+ specify the matrix), or use "convert format" and "alias format" when it comes to External Filters.

In case with direct access to file there isn't that much to do, Virtual Dub does conversion to 32b RGB in "full processing" and then later encoder does another conversion into YUV420 again. Nothing can be done here, fine detail loss quite noticeable even with lossess codes (Lagarith, Huff, x264 lossess mode) but whatever.

My main concern is case with frameserver. Would it make more sense to send output to VirtualDub through FS in 32b RGB or the final result won't be very different? My main concern primarily is to reduce colorspace conversion as mcuh as possible.


No benefit. That "32bit" RGB is 8bit per channel. 8 red, 8 blue, 8 green, + 8 "dummy" alpha . It's the same as the dmfs RGB output from vegas - there is no loss at that stage. All the loss occurs when you convert to YUV and subsample to 4:2:0.

[raffriff42]

Virtual Dub does conversion to 32b RGB in "full processing" and then later encoder does another conversion into YUV420 again. Nothing can be done here, fine detail loss quite noticeable even with lossless codesUse the FFmpeg input driver (http://forums.virtualdub.org/index.php?act=ST&f=2&t=22269&st=0#entry95795); it will preserve Fraps' native YV12, and does a better job of color resampling for sharper color details. V-dub will handle the conversion from full-range YUV automatically - there's no need for explicit conversion.

Set Video, Color Depth, Decompression to "Autoselect" and Output format to "Same as decompression format." As long as you don't use any filter that requires RGB (Filters dialog, enable Show Image Formats) YUV will be preserved from end to end.

[Kein]

Use the FFmpeg input driver (http://forums.virtualdub.org/index.php?act=ST&f=2&t=22269&st=0#entry95795); it will preserve Fraps' native YV12, and does a better job of color resampling for sharper color details. V-dub will handle the conversion from full-range YUV automatically - there's no need for explicit conversion.

Set Video, Color Depth, Decompression to "Autoselect" and Output format to "Same as decompression format." As long as you don't use any filter that requires RGB (Filters dialog, enable Show Image Formats) YUV will be preserved from end to end.

Cheers, that's pretty awesome, didn't know Vdub has ffmpeg plugin.

But what about FrameServer then? I often use mixed media in Vegas, could be fraps, could be recording in completely different codec with his own weird colorspace. What would be best option to feed through DebugMode FS?

I'm also curious about the case when I would need to the process vice versa in case with latter - not from Vegas and then final render with crop, but first the filters and then feed to Vegas for heavy editing. I assume Vdub's frameserver is useless here since Vegas does not understand it, nor he can read AVS. My only way is to use old and slow VFAPI with RGB24?

[poisondeathray]

But what about FrameServer then? I often use mixed media in Vegas, could be fraps, could be recording in completely different codec with his own weird colorspace. What would be best option to feed through DebugMode FS?


Best option for dmfs is RGB, because internally that is what vegas is using


I'm also curious about the case when I would need to the process vice versa in case with latter - not from Vegas and then final render with crop, but first the filters and then feed to Vegas for heavy editing. I assume Vdub's frameserver is useless here since Vegas does not understand it, nor he can read AVS. My only way is to use old and slow VFAPI with RGB24?

For importing into vegas, you could use avisynth virtual file system (AVFS)

But timeline performance can be really crappy , especially if you have lots of filters / slow avs scripts

[Kein]

But timeline performance can be really crappy , especially if you have lots of filters / slow avs scripts
Only crop + small resize (+30-35% of height). Should be fine then?

[poisondeathray]

yes should be fine - unless you have difficult footage (HEVC UHD :) or something) , slow hardware etc...

[Kein]

As long as you don't use any filter that requires RGB (Filters dialog, enable Show Image Formats) YUV will be preserved from end to end.
Seems like cropping requires RGB32 oh well

[sneaker_ger]

Make sure you have a YUV source opened, the image format column shows the currently used format, not all possible ones.

[Kein]

Yeah, well, it did not work very well:

http://i.imgur.com/TxeOBon.png

Opened FRAPS one with FFMPEG, set the Color Depth as rariff suggested.

UPD:

Okay, so if I open FRAPS recording directly in Vdub with FFMPEG plugin, it will detect color space properly YV12, yuvj420p, but won't let me go with it through VFW to x264.

However, if I open it through FFMPEGSource2 plugin in AviSynth, Vdub opens the avs through ffmpeg plugin with colorspace yuv420p and allows me to encode it.

UPD2:
Okay, so the range is still broken. I didn't do forced conversion to 4:2:0 through x264vfw this time, used "keep input colorspace", but the final results seems to be full/pc range.

So, the conversion is unavoidable anyway? I don't understand anymore what I'm doing wrong

[raffriff42]

Okay, so if I open FRAPS recording directly in Vdub with FFMPEG plugin, it will detect color space properly YV12, yuvj420p, but won't let me go with it through VFW to x264.Try output colorspace = "4:2:0 planar (YV12)."
I may have steered you wrong with "same as decode" :(
Opening through Avisynth, yes you have to do a range conversion.

[Kein]

Try output colorspace = "4:2:0 planar (YV12)."
I may have steered you wrong with "same as decode" :(
Opening through Avisynth, yes you have to do a range conversion.

Nope, in this case luma completely broken, way too lighter than original.

[Kein]

Okay, I'm losing my mind here. I don't understand what is going on anymore. Let's leave FRAPS alone and use another example:

I have YV12 source, it was recorded like that (dxtory YUV420(2x2), should be YV12 because U/V positions match each other, otherwise colors would be fckd). I can open it through AVIsynth and ffmpeg plugin and then feed video with correctly detected colorspace to AVSproxy-> AVidemux (again, as an example)

But here is a thing: unless I forcefully specify "ConvertToYV12(matrix="rec709")" for AVSscript for Avidemux, the result colorspace/range would be fucked and will be darker. But why would I need to do such conversion in a first place if I already have proper input colorspace? It drives me mad.

Get 2 apples, convert 2 apples into 2 apples, and share 1 apple with a friend.

[vivan]

But why would I need to do such conversion in a first place if I already have proper input colorspace? It drives me mad.UseInfo()before/instead of convert to check this.

[Kein]

I do, constantly. I know that Vdub+ffmpeg plugin and Avisynth+ffmpegsource2 detects the colorspace correctly. In case with AVSproxy for AVidemux -- it only works in YV12.

[poisondeathray]

But here is a thing: unless I forcefully specify "ConvertToYV12(matrix="rec709")" for AVSscript for Avidemux, the result colorspace/range would be fucked and will be darker. But why would I need to do such conversion in a first place if I already have proper input colorspace? It drives me mad.




How are you determining this ? or provide more details in your exact steps

The preview within avidemux isn't correct , this is known issue

I just tried this with avsproxy gui and it's bit for bit identical (tested a amplified differences compare script with exported lossless video). I used ffms2 with the custom tab in avsproxygui, and external script

Then I repeated it with ConvertToYV12(matrix="rec709") for the avsproxy. No differences detected - decoded image is bit for bit identical. And there shouldn't be as expected because it's a no -op

[Kein]

No differences detected - decoded image is bit for bit identical.
How did you do comparison? Just curious.

[poisondeathray]

How did you do comparison? Just curious.

Overlay(clip1, clip2, mode="Difference", pc_range=true)
Levels(127, 1, 129, 0, 255, false)

There are many variations you can use to verify, with subtract, overlay, psnr, ssim - even non avisynth methods

[Kein]

Is it supposed to look like that with 1:1?
http://i.imgur.com/B5yYaHy.png

[poisondeathray]

Is it supposed to look like that with 1:1?
http://i.imgur.com/B5yYaHy.png

Nope - This indicates there differences

[Kein]

Is it supposed to show such massive difference when encoded from original "lossless" YV12 to H.264/AVC1 in the same colorspace (no colorspace conversion forced) because of encoder? The 1:1 result will be only in case with lossless, right?

[poisondeathray]

Is it supposed to show such massive difference when encoded from original "lossless" YV12 to H.264/AVC1 in the same colorspace (no colorspace conversion forced) because of encoder? The 1:1 result will be only in case with lossless, right?

The 1:1 case only occurs with losses encoding in the same colorspace. It should show 1 shade of grey, or Y=128, U=128, V=128 . A higher quality encode will look more closer to 1 shade of grey than a lower quality encode - those losses will be from lossy compression

The Levels(127, 1, 129, 0, 255, false) amplifies the differences . You can comment that out if you only want to see the "normal" differences

[Kein]

Okay, gotcha, let me run some more tests.

[StainlessS]

Wrapped up.

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}

The show background thing just allows you to see exactly where the differences are by showing a dimmed version, best used with amp=true.

[Kein]

There is been an AVS script floating around that would allow you to analyze real-time color range/levels to see if it is full or limited, do you by any chance remember where it was or who made it?

I accidental lost it with HDD. Damn it was useful.

[raffriff42]

yeah, it's super-useful(IsRGB) ? RGBAdjust(analyze=true) : ColorYUV(analyze=true)

[Kein]

No, not this, this does not give me what I want.

Damn, basically, there was some article with general tips "how to upload videos on YT properly" and in that article the author dropped a link to another one with color range explanation, which was even more lengthy than original. That article contained pretty great script as well but I can't find it now. Probably that resource no longer exist it was around 2014

[StainlessS]

For yuv, histogram(mode="levels")

Hope I got that right, mobile.

Edit, there are other modes.

http://avisynth.nl/index.php/Histogram

[Reel.Deel]

There is been an AVS script floating around that would allow you to analyze real-time color range/levels to see if it is full or limited, do you by any chance remember where it was or who made it?

I accidental lost it with HDD. Damn it was useful.

There's OutRange (http://avisynth.nl/index.php/OutRange), this script scans and logs out of TV range frames. There's also ShowOverRange (http://avisynth.nl/index.php/ShowOverRange), this plugins paints pixels which are out of TV range. That's the the only filters I know of, if you find the script/article you're talking about please share.

[Kein]

Thanks Reel these are superuseful. I will try to find the script yeah.