Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th February 2016, 21:46   #1  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Conversion of YV12 to RGB

Hi,

I've just got VapourSynth up and running in linux (Kubuntu 15.10) and I'm now looking at the possibility of replicating some of the scripted AVISynth routines I use.

For conversion of my YV12 (HD AVC) sources to RGB and back I've been using what I call "Gavino's approach" as per:

http://forum.doom9.org/showthread.ph...88#post1571388

So:

Code:
ConvertToYV24(chromaresample="point", interlaced=false)
MergeChroma(PointResize(width, height, 0, 1))
ConvertToRGB32(matrix="rec709", interlaced=false)
...for further processing in RGB32 and then:

Code:
ConvertToYV12(chromaresample="point", matrix="rec709")
Is there a way the same can be done in VapourSynth?
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 7th February 2016, 22:06   #2  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,002
I have to ask why you need to convert to RGB and back at all in the first place. You probably shouldn't do that.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 8th February 2016, 00:56   #3  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Well, because:

Quote:
Originally Posted by WorBry View Post
...for further processing in RGB32...
And by that I mean encoding to intermediate RGB formats suitable/desirable for further processing in other linux applications (KDenLive, Cinelerra, Natron etc).

Basically, I want to examine the pros and cons of 'direct' transcoding of my YV12 sources to various RGB formats (UTVideo, Raw, PNG Image Sequences etc) with FFMPEG versus pre-conversion to RGB using Vapoursynth and piping out to FFMPEG for encoding. And then of course converting back to YV12 for encoding out to deliverable formats.

That's just one of the scenarios. The peculiarities of the "Progressive Segmented Frame" (PsF) AVCHD.mts format that some of my sources are recorded in and the way they are handled weigh into that evaluation. And another thing I want to look at is compression of full scale 0-255 sources to 16-235 before conversion to RGB. So there are good reasons.

I just would like to know if the above method that I use for converting YV12 to RGB with AVISynth has an equivalent in VapourSynth.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 8th February 2016 at 01:16.
WorBry is offline   Reply With Quote
Old 8th February 2016, 01:12   #4  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,073
Don't process in RGB if your source is YUV. It is almost certainly avoidable. That said, the builtin resizers should do the right thing if you really do need RGB, there's no need to try to manipulate chroma sample location manually.

As an aside, I'm really curious about what peculiarities these sources of yours have.
TheFluff is offline   Reply With Quote
Old 8th February 2016, 02:28   #5  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Well I didn't really want to go off on a tangent on this. But there are circumstances where it may be more desirable to pre-convert to RGB, and have control over the way that conversion is carried out.

KDenLive, for example, handles the YUV ranges of different source formats in different ways and depending on what effects are applied. In some cases the source YUV scaling is preserved, in others it is clamped or compressed (as in the case of full scale 0-255 sources) to limited 16-235 ranges. All in turn related to the way MLT, and thence FFMPEG, sets and respects the pix_format flagging. A whole subject in itself that I don't want to get into here. Suffice it to say that there are situations where using RGB input (and render) formats is expedient, especially when is it certain that the edit workflow will involve internal up-sampling to RGB and Rec709 conversion back to YUV anyway.

The reason why I use the above method for YV12<>RGB inter-conversion in AVISynth is precisely to minimize quality loss.

As for the "peculiarities" of the 1080/30PsF format that my AVCHD camcorder records in. I was merely referring to the fact that some NLE's still interpret this format incorrectly i.e. they treat the progressive YV12 chroma subsampling pattern as being interlace pattern, resulting in CUE artifacts. Also some source filters, like ffms2 (in AVISynth and VapourSynth) output the YV12 at double-frame rate (as duplicated frames) unless the correct fps is specified by the fpsden and fpsnum parameters. That's all I meant by "peculiarities"...."nuances to be aware of" might be a better term.

In answer to my question then - which is tantamount to "what is the most precise way to convert progressive YV12 to RGB32 with VapourSynth ?" - your advice is that I best stick to the "built-in resizer"? So is this referring to fmtconv, which I'll have to get my head around:

http://forum.doom9.org/showthread.php?t=166504

....or "something else" ?

Edit: So according to the fmtconv readme that would be:

YUV 4:2:0 to RGB:

Code:
c = core.fmtc.resample (clip=c, css="444")
c = core.fmtc.matrix (clip=c, mat="709", col_fam=vs.RGB)
c = core.fmtc.bitdepth (clip=c, bits=8)
and for RGB to YUV 4:2:0

Code:
c = core.fmtc.matrix (clip=c, mat="709", col_fam=vs.YUV, bits=16)
c = core.fmtc.resample (clip=c, css="420")
c = core.fmtc.bitdepth (clip=c, bits=8)
Correct ?

Edit2 - Ah, OK having tested that it appears that the only way to pipe to FFMPEG is via yuv4mpegpipe which would require conversion of the RGB back to YUV, which is clearly a redundant exercise. So I'm probably just as well using FFMPEG to transcode to RGB then or still doing it with AVISynth and vfw codecs?

What I might want to look at though is using VapourSynth for compressing full scale 0-255 YV12 sources (DSLR HD AVC clips) to 16-235 range. This is how KDenLive handles native full-scale sources, but I'm seeing a lot of luma "banding" in the MLT compressed YV12 and YUY2 renders
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 8th February 2016 at 06:22.
WorBry is offline   Reply With Quote
Old 8th February 2016, 11:44   #6  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,002
You can pipe raw RGB to ffmpeg just fine. You just have to manually specify width, height, pixfmt and all the other stuff it would otherwise read from the y4m header.

You can also simply use the built in core.resize.Bicubic(format=vs.RGB24, matrix_in_s="709") to do more or less the same thing.

Just try the resizers without tricks and see if they look good enough.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 8th February 2016, 12:57   #7  |  Link
age
Registered User
 
Join Date: Oct 2015
Posts: 41
The conversion of that script should be :
Code:
c = core.fmtc.resample (clip=c, css="444",kernel="point")                   #ConvertToYV24(chromaresample="point", interlaced=false)

                                                                              
r=core.fmtc.resample (clip=c,kernel="point",sw=0, sh=-1)                    #MergeChroma(PointResize(width, height, 0, 1))
c=core.std.ShufflePlanes(clips=[c,r], planes=[0, 1, 2], colorfamily=vs.YUV) #


c = core.fmtc.matrix (clip=c, mat="709", col_fam=vs.RGB)                    #ConvertToRGB32(matrix="rec709", interlaced=false)


c = core.fmtc.matrix (clip=c, mat="709", col_fam=vs.YUV)                    #
c = core.fmtc.resample (clip=c, css="420",kernel="point")                   #ConvertToYV12(chromaresample="point", interlaced=false)
c = core.fmtc.bitdepth(clip =c, bits=8)                                     #
For compressing full scale YV12 sources using expr:
Code:
bit=2**c.format.bits_per_sample-1         
c=core.std.Expr(clips=c, expr=[" {bit} x {bit} / 0.85882352941 * 1 / 0.06274509803 + * ".format(bit=bit),"  {bit} x {bit} / 0.50196078431 - 0.87843137254 * 1 / 0.50196078431 + * ".format(bit=bit),"  {bit} x {bit} / 0.50196078431 - 0.87843137254 * 1 / 0.50196078431 + * ".format(bit=bit)])
age is offline   Reply With Quote
Old 8th February 2016, 13:50   #8  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,115
Quote:
Originally Posted by WorBry View Post
The reason why I use the above method for YV12<>RGB inter-conversion in AVISynth is precisely to minimize quality loss.
Might be better to stick with traditional methods:
Quote:
Originally Posted by cretindesalpes View Post
YV12 <-> RGB conversions are not lossless whatever the bitdepth, unless you’re resampling the chroma planes with a specific kernel in order to make YV12 -> RGB -> YV12 lossess, but this may reduce the overall quality of each conversion.
Reel.Deel is offline   Reply With Quote
Old 8th February 2016, 13:54   #9  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Quote:
Originally Posted by Myrsloik View Post
You can pipe raw RGB to ffmpeg just fine. You just have to manually specify width, height, pixfmt and all the other stuff it would otherwise read from the y4m header.
OK thanks, that's good to know. Whilst searching for information I came across this post and so just assumed it was a non-starter:

http://forum.doom9.org/showpost.php?...85&postcount=5

Could you point me to any documentation that explains/shows how to do that? Trying to find pertinent information in the FFMPEG labyrinth is often like searching for a needle in a field of haystacks - a snippet here, a bug ticket there.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 8th February 2016 at 14:02.
WorBry is offline   Reply With Quote
Old 8th February 2016, 14:06   #10  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Quote:
Originally Posted by age View Post
The conversion of that script should be :
Code:
c = core.fmtc.resample (clip=c, css="444",kernel="point")                   #ConvertToYV24(chromaresample="point", interlaced=false)

                                                                              
r=core.fmtc.resample (clip=c,kernel="point",sw=0, sh=-1)                    #MergeChroma(PointResize(width, height, 0, 1))
c=core.std.ShufflePlanes(clips=[c,r], planes=[0, 1, 2], colorfamily=vs.YUV) #


c = core.fmtc.matrix (clip=c, mat="709", col_fam=vs.RGB)                    #ConvertToRGB32(matrix="rec709", interlaced=false)


c = core.fmtc.matrix (clip=c, mat="709", col_fam=vs.YUV)                    #
c = core.fmtc.resample (clip=c, css="420",kernel="point")                   #ConvertToYV12(chromaresample="point", interlaced=false)
c = core.fmtc.bitdepth(clip =c, bits=8)                                     #
For compressing full scale YV12 sources using expr:
Code:
bit=2**c.format.bits_per_sample-1         
c=core.std.Expr(clips=c, expr=[" {bit} x {bit} / 0.85882352941 * 1 / 0.06274509803 + * ".format(bit=bit),"  {bit} x {bit} / 0.50196078431 - 0.87843137254 * 1 / 0.50196078431 + * ".format(bit=bit),"  {bit} x {bit} / 0.50196078431 - 0.87843137254 * 1 / 0.50196078431 + * ".format(bit=bit)])
Brilliant. Thanks for that.
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 8th February 2016, 14:13   #11  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Quote:
Originally Posted by Reel.Deel View Post
Quote:
Originally Posted by cretindesalpes View Post
YV12 <-> RGB conversions are not lossless whatever the bitdepth, unless you’re resampling the chroma planes with a specific kernel in order to make YV12 -> RGB -> YV12 lossess, but this may reduce the overall quality of each conversion.
Seems like contradiction in terms, at face value. Still, once I've got the RGB to FFMPEG piping sorted, it would be interesting to see if there is anything to be gained in terms of quality preservation. It's how we learn isn't it....test/evaluate.
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 8th February 2016, 14:15   #12  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 723
Quote:
Brilliant. Thanks for that.
Maybe brilliant, but needlessly complicated.

Don't let ffmpeg (swscale) do any format conversions. VapourSynth's resizers (zimg) or fmtconv will do it better.

As for converting to RGB and back like in your initial post:
Code:
rgb24 = core.resize.Point(yv12, format=vs.RGB24, matrix_in_s="709", chromaloc_in_s="top_left")

# rgb processing

yv12 = core.resize.Point(rgb24, format=vs.YUV420P8, matrix_s="709", chromaloc_s="top_left")
There is no need to shift the chroma like in Gavino's code because resize.Point to yv24 followed by resize.Point back to yv12 is a no-op, provided you use the same chroma location in both calls.

If your yv12 input is full range, add range_in_s="full". You don't have to convert it to limited range yv12 first. But if you must convert to limited range yv12:
Code:
yv12_limited = core.resize.Point(yv12_full, range_in_s="full", range_s="limited")
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 8th February 2016, 14:33   #13  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Ah you're probably right. You guys are the experts on this stuff, and I wouldn't want to discard the advice given to me...again

"Sufficient today is the getting the RGB FFMPEG pipe to work thereof"
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 8th February 2016, 15:23   #14  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 723
Quote:
Originally Posted by WorBry View Post
"Sufficient today is the getting the RGB FFMPEG pipe to work thereof"
You can pipe planar RGB and change the plane order for ffmpeg's "gbrp" pix_fmt:
Code:
clip = core.resize.Point(clip, format=vs.RGB24, <options>)
clip = core.std.ShufflePlanes(clip, planes=[1,2,0], colorfamily=vs.RGB)
clip.set_output()
Code:
vspipe script.py - | ffmpeg -f rawvideo -pix_fmt gbrp -s 1920x1080 -r 24 -i pipe: -vcodec utvideo utvideo.mkv
Alternatively, you can pipe packed RGB and use ffmpeg's "bgr0" pix_fmt:
Code:
clip = core.resize.Point(clip, format=vs.COMPATBGR32, <options>)
clip.set_output()
Code:
vspipe script.py - | ffmpeg -f rawvideo -pix_fmt bgr0 -s 1920x1080 -r 24 -i pipe: -vcodec utvideo utvideo.mkv
The latter should be slightly slower due to the packing step. (ShufflePlanes is practically free.)
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 8th February 2016, 17:48   #15  |  Link
age
Registered User
 
Join Date: Oct 2015
Posts: 41
Reading the discussion linked in first post,that script is based on the assumption that avisynth's point resize can does "lossless" yv12->yv24->yv12 conversion only shifting the chroma, at least the point resize found in fmtc is the same.
From the documentation:
"point" Nearest neighbour interpolation. Same as Avisynth’s PointResize.

I admit that I don't know very well how to use zimg,
can I ask here how to upsampling 4:2:0 to 4:4:4 with the internal resizer?
age is offline   Reply With Quote
Old 8th February 2016, 18:18   #16  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,073
The moral of this story: Avisynth has a lot of old cruft and weird workarounds for odd problems that has gathered over the years since it hasn't had any development of note in many years and almost everything needs to be backwards compatible all the way to 2001. VS endeavors to do the Right Thing and doesn't hesitate to rethink things. Don't assume that just because you have to work around an issue in Avisynth that that same problem exists in VS (in this case, the VS resizers support custom chroma location settings).

Last edited by TheFluff; 8th February 2016 at 18:21.
TheFluff is offline   Reply With Quote
Old 8th February 2016, 22:59   #17  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Quote:
Originally Posted by jackoneill View Post
You can pipe planar RGB and change the plane order for ffmpeg's "gbrp" pix_fmt:
Code:
clip = core.resize.Point(clip, format=vs.RGB24, <options>)
clip = core.std.ShufflePlanes(clip, planes=[1,2,0], colorfamily=vs.RGB)
clip.set_output()
Code:
vspipe script.py - | ffmpeg -f rawvideo -pix_fmt gbrp -s 1920x1080 -r 24 -i pipe: -vcodec utvideo utvideo.mkv
Alternatively, you can pipe packed RGB and use ffmpeg's "bgr0" pix_fmt:
Code:
clip = core.resize.Point(clip, format=vs.COMPATBGR32, <options>)
clip.set_output()
Code:
vspipe script.py - | ffmpeg -f rawvideo -pix_fmt bgr0 -s 1920x1080 -r 24 -i pipe: -vcodec utvideo utvideo.mkv
The latter should be slightly slower due to the packing step. (ShufflePlanes is practically free.)
Thanks for that. Much appreciated.
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 8th February 2016, 23:01   #18  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Quote:
Originally Posted by TheFluff View Post
The moral of this story: Avisynth has a lot of old cruft and weird workarounds for odd problems that has gathered over the years since it hasn't had any development of note in many years and almost everything needs to be backwards compatible all the way to 2001. VS endeavors to do the Right Thing and doesn't hesitate to rethink things. Don't assume that just because you have to work around an issue in Avisynth that that same problem exists in VS (in this case, the VS resizers support custom chroma location settings).

LOL. Thanks. I'll bear that in mind as I haul my little row boat up onto the beach of this brave and enchanting new world.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 8th February 2016 at 23:07.
WorBry is offline   Reply With Quote
Old 15th February 2016, 04:17   #19  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,181
Regarding:

Quote:
Originally Posted by jackoneill View Post
You can pipe planar RGB and change the plane order for ffmpeg's "gbrp" pix_fmt:
Code:
clip = core.resize.Point(clip, format=vs.RGB24, <options>)
clip = core.std.ShufflePlanes(clip, planes=[1,2,0], colorfamily=vs.RGB)
clip.set_output()
Code:
vspipe script.py - | ffmpeg -f rawvideo -pix_fmt gbrp -s 1920x1080 -r 24 -i pipe: -vcodec utvideo utvideo.mkv
Alternatively, you can pipe packed RGB and use ffmpeg's "bgr0" pix_fmt:
Code:
clip = core.resize.Point(clip, format=vs.COMPATBGR32, <options>)
clip.set_output()
Code:
vspipe script.py - | ffmpeg -f rawvideo -pix_fmt bgr0 -s 1920x1080 -r 24 -i pipe: -vcodec utvideo utvideo.mkv
The latter should be slightly slower due to the packing step. (ShufflePlanes is practically free.)
I've had no problem running these scripts and the pipe commands with 'native' progressive HD AVC sources, but I've got an issue with AVCHD.mts clips from my Canon HFG10 camcorder which I mentioned earlier records in 1080/30PF format:

Quote:
Originally Posted by WorBry View Post
As for the "peculiarities" of the 1080/30PsF format that my AVCHD camcorder records in. I was merely referring to the fact that some NLE's still interpret this format incorrectly i.e. they treat the progressive YV12 chroma subsampling pattern as being interlace pattern, resulting in CUE artifacts. Also some source filters, like ffms2 (in AVISynth and VapourSynth) output the YV12 at double-frame rate (as duplicated frames) unless the correct fps is specified by the fpsden and fpsnum parameters. That's all I meant by "peculiarities"...."nuances to be aware of" might be a better term.
Applying the above methods for converting to RGB32 or RGB24:

Code:
import vapoursynth as vs
core = vs.get_core()
clip = core.ffms2.Source("Path...../TestHFG10.mts", fpsnum=30000,fpsden=1001)
clip = core.resize.Point(clip, format=vs.COMPATBGR32, matrix_in_s="709", chromaloc_in_s="top_left")
clip.set_output()
Code:
import vapoursynth as vs
core = vs.get_core()
clip = core.ffms2.Source("Path...../TestHFG10.mts", fpsnum=30000,fpsden=1001)
clip = core.resize.Point(clip, format=vs.RGB24, matrix_in_s="709", chromaloc_in_s="top_left")
clip = core.std.ShufflePlanes(clip, planes=[1,2,0], colorfamily=vs.RGB)
clip.set_output()
When I load the scripts in VapourSynth Editor and open Preview, the Preview screen is blank and there is the log error message:
Quote:
Error getting the frame number 0:
Resize error: Resize: field-based video not supported
Running just the bare source script:

Code:
import vapoursynth as vs
core = vs.get_core()
clip  =core.ffms2.Source("Path..../TestHFG10.mts",fpsnum=30000,fpsden=1001)
clip.set_output()
The preview shows a lot of "interlace-like" and block artifacts:

http://i.imgur.com/VlRyN8l.jpg

I recall having similar issues, although not as severe as this, when using ffms2 in AVISynth, and I switched to using L-Smash for a short while before DGDecIM arrived on the scene.

So, unless there's remedy for this, a better option might be to try L-Smash source? Unfortunately, the "extra plugins" package that came with VapourSynth and other required packages from here:

https://launchpad.net/~djcj/+archive/ubuntu/vapoursynth

....didn't include the L-Smash plugin.

I'd be really grateful if someone could advise me where to source and how to install the L-Smash plugin on my system.

As I mentioned in the first post I'm running Kubuntu 15.10 (64 AMD) and now also Mint KDE 17.3 (64 AMD). In both VapourSynth installations the plugins got placed in /usr/lib/x86_64-linux-gnu/vapoursynth

Cheers.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 15th February 2016 at 04:46.
WorBry is offline   Reply With Quote
Old 15th February 2016, 05:09   #20  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,073
You didn't post a video sample so my first recommendation would be to 1) read and understand error messages and 2) field-separate it and see if it's actually interlaced or not (it might be MBAFF for all I know). If it is interlaced then well, ya gotta deinterlace it. If it's not, figure out how to weave it properly, I guess, because FFMS2 evidently figures it to be field-based and outputs it as such.

Last edited by TheFluff; 15th February 2016 at 05:11.
TheFluff is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.