ColorYUV changes black level, can't restore it

[florinandrei]

AviSynth 2.6 running in MeGUI.

The video is a timelapse of the Sun, captured with an astrophoto camera mounted on a telescope. Capture was done in raw format (SER), converted with PIPP to AVI / MPEG4. The AVI / MPEG4 file is then processed with AviSynth in MeGUI.

The script is just this:

LoadPlugin("C:\opt\megui\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("file.avi")

In the preview window I see this image, which matches very well what's in the SER and the AVI/MPEG4 files:

https://forum.doom9.org/attachment.php?attachmentid=16007&stc=1&d=1502777591

But the color balance was wrong from the very beginning (due to some filters on the telescope) and I need to fix it, so I do this:

LoadPlugin("C:\opt\megui\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("file.avi")
ColorYUV(off_u=-35, off_v=18)

And this is the result:

https://forum.doom9.org/attachment.php?attachmentid=16008&d=1502777590

The Sun looks right, but the background has changed frome pitch black to muddy brown.

Question 1: Why does this happen?

Question 2: How do I remove it? I've tried:

Levels(18, 1.0, 180, 0, 255, coring=false)

But it does not remove the brown background completely, it just makes it less obvious. Increasing the first parameter above 18 does not make a difference at first, and then makes everything much worse (the background turns red).

Please note that the Levels filter also stretches out the white level to make the whole image brighter, but that makes no difference for the brown background, which looks the same either way.

[StainlessS]

Well for one thing, would be best to try to use Avisynth via eg VirtualDubFilterMod or AvsPMod, or anything else other than MeGUI,
MeGui is fine if dealing with a working script but not for experimentation.

ColorYUV is the wrong tool for the job. Black/grey have U and V @ 128, u_off = -35, subtracts 32 from all values of u, so greys which should be u=128 come out as 128 - 35, ie not grey. In your Levels() thing, you are effectively trying to correct problems that were introduced by the ColorYUV step, best not create additional problems in the first place.

If filtered in telescope, I would assume that filters operated in RGB, and so should be corrected in RGB, using some RGB tweaker., eg Tweak.
[EDIT: Actually, come to think of it I think Tweak operates in HSL not RGB, so maybe give it a miss]
BuiltIn filters:- http://avisynth.nl/index.php/Internal_filters#Color_conversion_and_adjustment_filters
Maybe RGBAdjust or Tweak.

External filters:- http://avisynth.nl/index.php/External_filters#Levels_and_Chroma
Suggest RgbAdapt:- https://forum.doom9.org/showthread.php?t=170642
Has a still image grapher (eg needs BMAP/JPeg, [EDIT: Cant remember if it accepts png or not, think it depends upon system
natively accepting png])
https://s20.postimg.org/wxyq8spkt/Fishv3_zpsfjmoyerr.jpg (https://postimg.org/image/yd0axiqnt/)

You will [EDIT: may] need something like ConvertToRGB32 beforehand, and eg ConvertToYV12 afterwards (in the script)
and just ConvertToRGB32 when saving frame as bitmap for the grapher.

[poisondeathray]

Another approach is to mask out the "sun" using a luma key or similar techniques. (Basically separate "bright" from "dark")

[StainlessS]

Another approach is to mask out the "sun" using a luma key or similar techniques. (Basically separate "bright" from "dark")

I'm guessin' that if the op is using avs in MeGUI, then might need a bit of a hint on how to do that.

[poisondeathray]

I'm guessin' that if the op is using avs in MeGUI, then might need a bit of a hint on how to do that.



Right, I can't "see" the attachments, awaiting to be approved

But I remember the name. He asked about stabilization over at videohelp

I'm assuming it's this video, or something like this
https://www.youtube.com/watch?v=7NicwNXSdbs

I downloaded that video, cropped to the region of interest . It's not necessary to stabilize, but I did in this example as suggested in the other thread. You can only stabilize it "roughly" , because the sun edges aren't the same (there is wobbling and morphing). But if you decide to stabilize it, you can just use a static cutout mask (basically a circle) to isolate the sun, which is another approach. The basic idea is you want to separate the foreground (sun) from background (black) , so you can either replace the background or filter the layers separately without affecting each other.

Here is an example.
https://www.mediafire.com/file/0kr8qrh95qqiyx2/stabilized%2Clumamask.zip

1_stabilized.mp4 is the stabilized video
2_lumamask.mp4 is (1) but using a mask to replace the background with "black"

You can see in the stabilized video, the red/brown background superimposed on the black moving around, as the stabilizer adjusts the position/rotation of the sun layer . The "black" is actually transparent, but the alpha channel is discarded in the video

The technique used in this example is called a "luma mask". You are isolating elements based on "brightness" levels . There are many different variations, but here is a simple one based on a binary mask using masktools in avisynth


LSmashVideoSource("1_stabilized.mp4")
orig=last

#generate mask
orig
mt_binarize(40,false, u=-128,v=-128)
mt_expand(u=-128,v=-128) #expand the mask
blur(1) #soften mask edge a bit
msk=last

#generate black background, the same characteristics as "orig" the source clip
blk=blankclip(orig)

mt_merge(blk,orig,msk,true)


You want to view the mask and manipulate it with various settings / filters so it "covers" the area of interest (the sun) . 100% white means 100% coverage. Shades of grey are intermediate. 0% white (or 100% black) means not part of the mask. If you have a good mask, you can apply filtesr through the mask (so only foreground object gets affected without affecting background) , or you can replace the background entirely as we did here - it was just "black", but you could have replaced it with stars, the milky way, the death star, etc..... whatever

[StainlessS]

Knew you'de be back Poison, yours is no doubt better than mine, but here goes anyway.

Below is set up for a Moon shot rather than sun, I dont got no sun here.


# Requires GSCript (or avs+)

#ImageSource("D:\PlanetCrop\MOON\IMG_5245.jpg",end=0)
AviSource("D:\PlanetCrop\Moon2.avi")
#return last

ConvertToYV12

### CONFIG ### # Defaults for iCount(0) and dSz(1) are OFF, raise one or both, dSz probably more effective.
HI = 50 # Greater or equal is Sun (I dont have your test clip, but I do got a moon, need raise somewhat, maybe)
iCount = 0 # Inflate,deflate interation count (higher get rid of some raggedness)
dSz = 1 # Feathering, 1 <= dSz. (1=Off)

Mask=MakeMask(Last,Hi,icount=iCount,dSz=dSz)

Return Mask # Perusal, COMMENT OUT when happy with Mask

Sun = ColorYUV(off_u=-35, off_v=18) # Or whatever
Overlay(Last,Sun,Mask=Mask)
Return Last

Function MakeMask(clip c,Int Hi,Int "iCount",Int "dSz") { # ret Mask Luma Plane Only
c
iCount = Default(iCount,0)
dSz = Default(dSz,1)
Assert(dSz>=1,"MakeMask: 1 <= dSz")
YExpr="x " + String(Hi) + " < 0 255 ?"
Mt_Lut(yexpr=YExpr,U=-128,V=-128)
GSCript(""" # Comment out this line if AVS+
For(i=0,iCount-1) { Mt_InFlate(chroma="-128") }
For(i=0,iCount-1) { Mt_DeFlate(chroma="-128") }
""") # Comment out this line if AVS+
W=Width/(dSz*2)*2 H=Height/(dSz*2)*2
(dSz>1) ? GaussResize(W,H,p=1.0).GaussResize(c.Width,c.Height,p=1.0) : NOP
Return GrayScale
}

[Motenai Yoda]

Instead of mask the sun/moon, why not to use gain_u/gain_v or as mentioned before rgbadjust?

[StainlessS]

Instead of mask the sun/moon, why not to use gain_u/gain_v or as mentioned before rgbadjust?
Well obviously, that would be just too easy :)
EDIT: Note, first graphic posted was of RgbAdapt(external), not RgbAdjust(builtin).


Here, ColorYUV2 (not ColorYUV), also has graffer thing, can
also apply S-Power curve to chroma.

https://forum.doom9.org/showthread.php?t=156774&highlight=ColorYUV2

https://s20.postimg.org/47y26xhfh/CYUV-1-0b-frog-desat.jpg (https://postimg.org/image/3v6o0qz5l/)

EDIT: Above is for Beta @ Post 20, here the non beta showing S-Power curves.
https://s20.postimg.org/e05bqbhq5/Coloryuv2_Graf.jpg (https://postimg.org/image/r4aw309rt/)

EDIT: ColorYUV2 does not have Dither of Avs 2.6 ColorYUV.

EDIT: I guess with a ScriptClip, YPlaneMax(threshold=whatever), and Levels, could use the Y Plane itself as a mask.

EDIT: Daft try, it needs to be tested with a sun sequence

#ImageSource("D:\PlanetCrop\MOON\IMG_5245.jpg",end=0)
#AviSource("D:\PlanetCrop\Moon2.avi")
AviSource("D:\PlanetCrop\_tmp_video_1502839958370.webm.AVI")
ColorYUV(off_u=35, off_v=-18) # Un-Botch
ORG=Last
#return last

ConvertToYV12

Thresh = 0.0
IMin = 16
OMin = 16
OMax = 235
Gamma = 1.0

SSS="""
mx=YPlaneMax(threshold=Thresh)
Levels(IMIN,Gamma,mx,OMin,OMax,coring=false)
Return GrayScale
"""

Mask=Last.Scriptclip(SSS)

#Return Mask # Perusal, COMMENT OUT when happy with Mask

Sun = ColorYUV(off_u=-35, off_v=18) # Or whatever
Overlay(Last,Sun,Mask=Mask)

StackVertical(ORG,Last) # Comment OUT
Return Last

[StainlessS]

I've just seen PoisonDeathRay posted clip (1st one), I'm assume that it should originally look like this, ie Bluish ?


AviSource("D:\PlanetCrop\_tmp_video_1502839958370.webm.AVI")
ColorYUV(off_u=35, off_v=-18) # Undo the OP fix


We really prefer access to the broken clip, not the botched fix.

[StainlessS]

Should the sky be bluish (or Black) when fixed ?

Presume that filtering was maybe something like view through electric arc welders visor.

My 2nd clip is best of my two.

#ImageSource("D:\PlanetCrop\MOON\IMG_5245.jpg",end=0)
#AviSource("D:\PlanetCrop\Moon2.avi")
AviSource("D:\PlanetCrop\_tmp_video_1502839958370.webm.AVI")
Crop(1024,520,-600,-480) # Comment OUT
#Crop(640,400,-660,-500) # Comment OUT
ConvertToYV12
ColorYUV(off_u=35, off_v=-18) # Un-Botch
#StackHorizontal(Last,Last)
ORG=Last
#ShowChannels
#return last


Thresh = 0.0
IMin = 16
OMin = 0 # was 16, mask should be 0
OMax = 255 # was 235, mask should be 255
Gamma = 1.0

SSS="""
mx=YPlaneMax(threshold=Thresh)
Levels(IMIN,Gamma,mx,OMin,OMax,coring=false)
Return GrayScale
"""

Mask=Last.Scriptclip(SSS)

#Return Mask # Perusal, COMMENT OUT when happy with Mask

#Sun = ColorYUV(off_u=-35, off_v=18) # Or whatever
Sun = ColorYUV(off_u=-43, off_v=26) # Sun About as close to WHITE/GREY as we can get

Overlay(Last,Sun,Mask=Mask)
StackVertical(ORG,Last) # Comment OUT
Return Last


https://s20.postimg.org/bnk0cttz1/Solor.jpg (https://postimg.org/image/59ux9kp2x/)

EDIT: Changed to Sun = ColorYUV(off_u=-43, off_v=26) # Sun About as close to WHITE/GREY as we can get
Dont know if you wannit looking White/Grey, or a nice bright yella thing :)
Of course you could always just GrayScale() it instead of all that script.

https://s20.postimg.org/72xtxwa9p/Solor2.jpg (https://postimg.org/image/azb5tvv95/)

Below, result of 1st script. with more yellow tinge, original ColorYUV numbers (iCount and dSz both OFF)

https://s20.postimg.org/f9uf37pq5/Solor3.jpg (https://postimg.org/image/bq8hden09/)

EDIT: Below PoisonDeathRay script, but we first un-botched clip (corrected to equivalent of original source)
and then corrected again via mask. Sky is pure black.


AviSource("D:\PlanetCrop\_tmp_video_1502839958370.webm.AVI")
Crop(1000,520,-600,-480) # Comment out
ConvertToYV12
ColorYUV(off_u=35, off_v=-18) # Un-Botch
ORIG=Last

#generate mask
orig
mt_binarize(40,false, u=-128,v=-128)
mt_expand(u=-128,v=-128) #expand the mask
blur(1) #soften mask edge a bit
msk=last
#return msk
#generate black background, the same characteristics as "orig" the source clip
blk=blankclip(orig)
Sun = orig.ColorYUV(off_u=-35, off_v=18) # Added to PDR script as he was using bothed version (which we above un-botched)

mt_merge(blk,Sun,msk,true)

https://s20.postimg.org/vyvuz4mbh/Solor4.jpg (https://postimages.org/)

I Kinda like the 2nd near grey version, after all, the sun is only yellow because of our atmosphere (as is the [EDIT:day time] sky blue).
The Brightest input luma is about 153.

[florinandrei]

First off, thanks everyone, I've learned so much from you all!

The capture format is SER, which is raw sensor info in video format. I can't read SER with AviSynth, so with the PIPP app I convert it to either MPEG4 or Ut codec (AVI container either way). For some reason, when reading from MPEG4, AviSynth switches automatically to YUV mode; when reading from Ut, it switches to RGB mode. I can convert between modes anyway, so it's no big deal, but I was surprised by that at first.

I've decided to use Ut as the intermediate format, because it's lossless. As a bonus, AviSynth goes directly into RGB mode, which is what I need for the RGBAdjust() filter, so no need to do an extra switch in the script explicitly.

After some experimentation, I ended up with this script:

LoadPlugin("C:\opt\megui\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("file.avi")
# 4944 x 3284

crop(0, 0, 0, -404)
# 4944 x 2880

RGBAdjust(r=2.03, g=1.53, b=0.94)

AddBorders(88, 0, 88, 0)
# 5120 x 2880 (a.k.a. 5k)

ConvertToYV12()

The result looks much better, the background is solid black (or something like [2,2,0], close enough), and the color balance of the Sun is more natural (I still need to tweak it a little).

The YUV filter I've tried before was distorting not just the background, but also the color of the Sun near the edge. It's true that the color at the edge is different from the center (a little more yellow/orange/red), but the YUV filter was exaggerating it a lot. Now it's more discrete. See here:

https://forum.doom9.org/attachment.php?attachmentid=16009&stc=1&d=1502862020

It's also easier for me to think in the RGB space.

Using AvsPmod helped a lot to cycle quickly through testing successive changes to the script.

The reason for the initial blueshift while capturing is complex. I use a Baader solar film filter before the light enters the telescope, then I use a Baader UV/IR-block filter before the camera. Both filters are color-neutral. So the Sun looks almost white after all filters, when seen by the human eye. But the camera (ZWO ASI071MC-Cool) has a different reaction to the spectrum there, not entirely sure why (perhaps some residual UV leak). Anyway, it's easy to fix in RGB.

I'm guessin' that if the op is using avs in MeGUI, then might need a bit of a hint on how to do that.

That's a very good guess. :)

But I remember the name. He asked about stabilization over at videohelp

That was me, yes. I've solved it a different way. If I do drift alignment for 1 hour or so, the equatorial mount becomes polar-aligned precisely enough that the image of the Sun remains within the frame for the whole 2.5 hours capture session. So I don't need to do bumpy adjustments while capturing.

The video you've found on my YouTube channel is how it looks like after drift alignment. It's stable enough.

There's a slight residual drift, due to remaining polar alignment errors. There are slight hops and bumps in the drift - these are due to imprecision in tracking by the EQ mount. Then there are quick vibrations in drift, these are due to the wind pushing the telescope (after 60x frame rate acceleration). And finally the small scale shimmering and wavy motion of the Sun's edge is due to atmospheric turbulence (a.k.a. "seeing" in the jargon).

Thanks again everyone, you've solved a nasty problem, this is much appreciated!

[StainlessS]

Glad you're a happy bunny :)

Dont know if of interest, but see here PlanetCrop(), takes time lapse sequence of images (not video) of planetoid objects, and crops/resize/align
into a video clip. (there was requirement to cope with differing sized input images):- https://forum.doom9.org/showthread.php?p=1606025#post1606025
Edge feathering is more a requirement for non solar planetoids, eg moon in part phase (as in my first script, seems unnecessary for solar).

The script is updated in current_RT_Stats to v1.02. (RT_Stats first post of link above).

And just for good measure, (maybe you are one of those daring Lightning chasers), thread on extracting only lightning strikes from long video pointed at storm area.
https://forum.doom9.org/showthread.php?t=172639&highlight=lightning

Sorry, nothing on tsunami, earthquake, near earth miss nor planet killer impact events.

[florinandrei]

Thanks everyone for your help, I was able to take a timelapse of the total solar eclipse on August 21st 2017, processed the capture file using advice I've received here, and I like the results:

https://www.youtube.com/watch?v=KyLHTk69nC4

[wonkey_monkey]

I like the results:

So do I! Did you manually adjust the exposure during totality, or was it all on auto?

[lansing]

Ok this is kind of a late reply but I wanted to explain why this happen.

The YUV colorspace is brightness independence, so the changes in brightness (Y) will have no effect on the other two channels (UV).
Here's a simple example, YUV(0, 0, 0) will give you a black color, YUV(0, 130, 255) will also give you a black color, simply because there're no brightness, however that color of the other two channels are still there, you just didn't see them. So when you start increasing the brightness, the color that were once hidden will reveal. In YUV, what you see is black only means that it has no brightness, there isn't a term "true black" that we were used to. In your case, your original clip have a red cast hidden in the darker areas, so when you increase the brightness you start seeing the red.

So in conclusion, don't ever use YUV tool for color correction, they simply don't work, and the YUV parade is unreadable. It makes me cringe seeing all the amateur youtube videos that are teaching people to remove color cast with YUV tools in photoshop. RGB tools is the only preferred way (RGB curve) and you seems to find that out yourself through trials and errors.:)

[florinandrei]

So do I! Did you manually adjust the exposure during totality, or was it all on auto?

All auto on software side. I just manually removed / replaced the main filter at the beginning / end of totality. Takes only a second to do so.

Tech details including AviSynth script are in the YouTube video description.

Ok this is kind of a late reply but I wanted to explain why this happen.

The YUV colorspace is brightness independence, so the changes in brightness (Y) will have no effect on the other two channels (UV).
Here's a simple example, YUV(0, 0, 0) will give you a black color, YUV(0, 130, 255) will also give you a black color, simply because there're no brightness, however that color of the other two channels are still there, you just didn't see them. So when you start increasing the brightness, the color that were once hidden will reveal. In YUV, what you see is black only means that it has no brightness, there isn't a term "true black" that we were used to. In your case, your original clip have a red cast hidden in the darker areas, so when you increase the brightness you start seeing the red.

So in conclusion, don't ever use YUV tool for color correction, they simply don't work, and the YUV parade is unreadable. It makes me cringe seeing all the amateur youtube videos that are teaching people to remove color cast with YUV tools in photoshop. RGB tools is the only preferred way (RGB curve) and you seems to find that out yourself through trials and errors.:)

Makes sense and clarifies a lot of things for me. Thank you!

[StainlessS]

Thanx for posting the total eclipse, very impressive.

I only recall ever seeing a partial, about 10 years ago.
Went quite dark and all the birds went silent, very spooky.
Quite humbling, and lets one know without doubt, that we are little more than microbes in the soup.