Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th February 2019, 12:57   #1  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,010
Extracting frame(s) from video with stacking and denoise

It's years there are programs/techniques that stack multiple photos to have increased light and/or less denoised one (mosly in low light).

Is it possible to "stack" multiple frames from a video, correcting pan and zoom (sometimes rotation), and denoise to extract a specific frame or multiple frames as images?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 17th February 2019, 17:03   #2  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,096
With photos, perhaps you are talking about HDR (High Dynamic Range) where you combine identical photos, taken on a tripod a few seconds apart, but each taken with a different shutter speed. This is now done automatically, in a fraction of a second, with the push of a single button, in some cameras. It lets you capture pictures that are perfectly exposed in both the shadows and the highlights, even when there is extreme exposure difference. Example: a woman in a white wedding dress in direct sun standing next to a cave, with the rest of the wedding party ten feet back in the cave. With this technique they will all look perfectly exposed, even without the addition of any additional light. I suppose you might get some noise reduction as part of this averaging process, but most of the time it is used to achieve absurd dynamic exposure range. The photos often look a little strange.

I don't know of any video cameras that offer this.

Averaging multiple frames to reduce noise is how temporal denoisers work. These denoisers first take the adjacent frames and attempt to move and rotate them so they match, as well as can be done, the current frame. In theory, if you can match them perfectly, when you subtract them, all you are left with is the noise, and you can then delete that noise. You need both of the adjacent frames (i.e., three frames total) to do this because otherwise you don't know if the noise is in the adjacent frame or in the current frame.

Play around with MDegrain2 (part of MVTools2) if you want to see how it works.There are several other of these temporal denoisers as well.
johnmeyer is offline   Reply With Quote
Old 17th February 2019, 20:30   #3  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 4,091
Quote:
Originally Posted by johnmeyer View Post
With photos, perhaps you are talking about HDR (High Dynamic Range) where you combine identical photos, taken on a tripod a few seconds apart, but each taken with a different shutter speed.
Alternatively, AutoStakkert and/or Registax are excellent tools.

I use them for processing video footage of the moon/planets I shoot through my telescope. Here is an example of the footage from my camera (Nikon DSLR + Maksutov 1400/100 telescope + 2x focal extender) and here is the processed photo after AutoStakkert (used for stacking) / Registax (used for wavelet sharpening) / Photoshop (adjusting levels, etc.). As you can see, the video footage has a lot of atmospheric disturbance which stacking eliminates nicely.

These programs are mostly used for astronomy imaging but I used them for terrestrial purposes as well.

Last edited by Groucho2004; 18th February 2019 at 12:31.
Groucho2004 is offline   Reply With Quote
Old 17th February 2019, 20:33   #4  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 1,642
Quote:
I don't know of any video cameras that offer this.
I can think of one example I've seen in the past of a camera which recorded alternate frames at different exposures, and I'm sure there was another setup which used an optical splitter of some kind to record two different exposure videos through the same lens.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 18th February 2019, 12:36   #5  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,381
Quote:
I don't know of any video cameras that offer this.
Mobile phones have been offering HDR video recording for a few years, my xperia Z2 (~2014) has this, but cant say that I've ever used it. : https://www.gsmarena.com/sony_xperia_z2-6144.php

EDIT: Currently, Chinese manufacturers seem to be making loads of 10 core tablets and phones, likely lots of new capability coming.
EDIT: My mates more recent xperia XZ has some kind of laser whatsit, and can take 3D video which can be fed into a 3D printer
(tricky to slowly walk around item to capture, easier if you have some kind of rotating table with static camera position).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 18th February 2019 at 13:10.
StainlessS is offline   Reply With Quote
Old 25th February 2019, 18:45   #6  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,010
Nope. I am not talking about HDR.

I am talking about getting multiple shots of same thing, same exposure and stack to "average" noise and/or increasing brightness, such as this.

My final idea is for underwater based shooting, when sometimes light is never enough, or astronomy.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 25th February 2019, 21:26   #7  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,096
Quote:
Originally Posted by tormento View Post
Nope. I am not talking about HDR.

I am talking about getting multiple shots of same thing, same exposure and stack to "average" noise and/or increasing brightness, such as this.

My final idea is for underwater based shooting, when sometimes light is never enough, or astronomy.
I used to do this for video and wrote about it ages ago in this thread:

my-ultimate-vhs-tape-restoration-recipe

I can't remember now, but I think I posted the "recipe" for how you have to combine the images. If you do it wrong, you end up with exposure changes.

Also, I was capturing the same tape multiple times so you would think that aligning the multiple images would not be an issue.

Wrong.

Because of timing issues and also because of field alignment. Analog video doesn't store fields and sothere is no "top" and "bottom" and therefore one or more of the captures often started with a different field.

In your case, alignment is also going to be an issue if there is any variation between the captures. I now fully understand that you are not doing HDR, but the issues you will face are identical. Any movement in the scenery (trees moving, people walking, etc.) will screw up the result. Any movement in the camera will reduce the effectiveness. For underwater photography, both the camera and almost everything in the scene is constantly moving, so even if you take all your pictures in 1/4 of a second, which would be pretty fast, there will be lots of changes between frames.

While everyone has their own budget, I would suggest that if your still photos are particularly noisy, you might want to consider one of the high-end Canon or Nikon DSLRs. I have seen some of the results from extremely high ISO (2000-10000) and the noise is unbelievably low. At normal ISOs and normal exposures, it is non-existent.

In addition, another suggestion is to invest in Adobe Lightroom, shoot in RAW, and then use Lightroom's amazing tools to improve your shots. I got this advice from someone who has since become a major-league professional photographer, and it dramatically improved the results I was able to get.

BTW, if your main interest is underwater photography, there is an expatriate American, living in Thailand, who used to help me a lot over in the Vegas forum, when I was active there. Check out not only his web site, but his extensive writings on the tools and techniques he uses to achieve his world-class results:

Nick Hope Underwater Photography
johnmeyer is offline   Reply With Quote
Old 27th February 2019, 11:05   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,381
At a guess, I think that tormento may be considering [at least in part] 're-creating' the missing red part of the spectrum which is absorbed by water when at significant depth.
(guess based on past thread, and this quote "sometimes light is never enough")
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 27th February 2019, 11:13   #9  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,010
Quote:
Originally Posted by StainlessS View Post
At a guess, I think that tormento may be considering [at least in part] 're-creating' the missing red part of the spectrum which is absorbed by water when at significant depth.
(guess based on past thread, and this quote "sometimes light is never enough")
That would be gold but I think almost impossible

It could be simulated only applying a reverse water wavelenght absorption formula and only at a given depth and given distance from object. The first part is partly done by devices such as Paralens camera, The distance estimation is currently an impossible to have data (unless parallax blah blah..).

What I'd like to do is to extract multiple frames from a video, motion compensate them and stack such as done in astrophotography, to enhance brightness and dynamic. Please notice that in diving, such as other type of video shooting, panning is really limited and you sometimes have multiple seconds of video on same subject.

With a limited budget, what amateur divers do is to have a low end camera with an expensive diving frame. Sometimes a Gopro, that goes down to 60 meters.

That is my current equipment and, in certain situation, is really hard to switch from video to photo mode with thick gloves or case mounted over head.

The Protune material you can get from latest iteration of Gopro is really hgh quality material but the small sensor limit the amount of collected light.

You can shoot at 4k/30 fps or in good light situation even at 60 fps but lowering ISO to more than 800 is too noisy.

Thus my idea of trying to apply the photo stacking idea to video.

But this could also apply to video surveillance (where video is almost still), night shooting and other fields.

And, please, remember that what I want as output is one photo shoot from a small frame sequence. I think that as a first step would be a great goal.

If any red channel information could be recovered, much better. But sometimes you have NO red information.
__________________
@turment on Telegram

Last edited by tormento; 27th February 2019 at 11:25.
tormento is offline   Reply With Quote
Old 27th February 2019, 11:38   #10  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,381
T,
Maybe give Introspected (not on-line since 4 Feb) a poke, see if he has any suggestions, his thread "AutoOverlay" here:- https://forum.doom9.org/showthread.php?t=175247
[its in the devs forum]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 1st March 2019, 08:27   #11  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,068
Quote:
Originally Posted by tormento View Post
If any red channel information could be recovered, much better. But sometimes you have NO red information.
You may want to check out 3D LUT Creator for this:
https://youtu.be/gHx0a4TKQbo?t=27
lansing is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.