true HDR video

[jmac698]

HDR means High Dynamic Range, and it refers to keeping detail in bright/dark areas. Normally pointing a camera at a light bulb you just washed out white; with HDR you could also see the filament.
Now, there's filters which simulate this, but real HDR only comes from more than 8 bits per channel.
You could do a multiple pass capture of a video source with different brightness settings of your capture card to get true HDR. This is popular in digital photography where multiple exposes 1/2 F-stop apart are combined in a program to output an HDR file; which can then be adjusted back down to 8bits with preserving the shadow and highlight details as desired.
The same could be done for video purely in scripting or a plugin. The storage would have to be done in two clips to store all 16bits.

However, since original footage is not usually HDR, this has little advantage in capturing. One possibility is an old VHS recording from a real analogue camera; ironically this probably has HDR information. A digitally produced source has already been digitized and the best you can do is recover the original bytes; a true analog source (analog tube camera connected to analog tape) has never been digitized, and you could then digitize in HDR for improved quality.

You would cap same footage from tape at different brightness settings, use thresholding to make a mask, use the mask to replace white pixels with pixels from 2nd clip, then 1st clip is dark pixels and 2nd clip is bright pixels, but each with it's own processing for improved contrast, with one common video level in between (also the thresh value you set). Make sense?

[mitsubishi]

Is this a question or a rant?
HDR in video games really owns, there's nothing like coming over a hill in NFS: MW and being blinded by the sun :cool:

EDIT: oh I get it, sorry didn't see what forum it was in:rolleyes:

[Mug Funky]

analog sources are still clipped to fit in broadcast dynamic range. there may be some overshoot, but a) there'd be no useful information there, and b) it'd be more easily captured by just turning the contrast down.

so far there's no cameras that can capture in the kind of dynamic range that HDR enthusiasts grab pictures in (generally one is supposed to keep taking photos at ever higher f-stops until there's no white left... and at the other end keep opening up the aperture until there's no black left).

i'd love to see some kind of sensor that can grab floating point, but i doubt that sort of thing will exist any time soon.

[Blue_MiSfit]

I've been messing with HDR still images, and they're a lot of fun. Even if you could specially shoot the same video several times with different settings and merge, the processing time involved would be insane. Merging just 5 RAW images from an EOS 10D takes ~ 5 minutes on my computer, and then you have to run the curves to pull it down to 16 bits / channel - so that your monitor can actually display everything :)

[davidhorman]

Nice use of HDR:

http://www.fieldofview.com/spv/show.php?file=adr-downing.xml

David

[jmac698]

I will just have to test this. I can try with an analog live camera and of course, recording that to analog tape is the same thing.

David: nice page!

Mug: I was thinking in the dark end, actually. It's very noisy down there, and getting more than 8bits depends on the dynamic range of the sensor in the camera, ultimately.
A floating point sensor makes no sense. There's various types of Analog to Digital converters, but they all rely ultimately on power of 2 steps in measurement. There are 16bit light sensors, like the one in your scanner, hence a 48bit color scanner. The scanner driver does HDR for you when you set brightness and contrast.

[rfmmars]

so far there's no cameras that can capture in the kind of dynamic range that HDR enthusiasts grab pictures in

Well there are several but they are too slow for action video. There use is in survailence video. A three chip camera could be modified for real time HDR.

HDRagc is the best bet at the this time.

Richard
photorecall.net

[Backwoods]

www.red.com

Records video like a RAW DSLR file. You can make separate exposure files and merge, would take along time though. HDRAGC is the best bet as of now.

[jmac698]

The Corpse Bride was shot with a EOS 1D Mark II camera. The raw files are more than 8bits. This probably makes it one of the best quality digital movies of real objects.
http://en.wikipedia.org/wiki/Corpse_Bride
http://en.wikipedia.org/wiki/Canon_EOS-1D_Mark_II

[morsa]

I've been waiting for an avisynth slow migration to higher than 8 bits and YUV 4:4:4 for years. Anyway I don't think that will ever happen.At least not before everything goes higher than 10 bits.
DigiBeta (despite what Shodan may say.Sorry Shodan) is indeed 10 bits, so it has 1024 levels of grey.That was confirmed to me by Graemme Natress , Red team) and if I don't remember wrong I had some comments about why not going to at least 10 bit precision in avisynth.

The corpse bride and Any digital film you see ends in 10 bit log scale (usually the Cineon Log scale). May be sensors are 14 bit or the like, but they are linear.So unless you apply some kind of gamma correction it will be quite difficult for you to see anything from it.

[jmac698]

A CCD measures accumulated current in each CCD cell. The current is created by the photoelectric effect. (current can easily be converted to voltage drop).
http://www.ghg.net/akelly/ccdbasic.htm
It has a linear response to light
http://en.wikipedia.org/wiki/Photodiode
ADC's measure linearly, and your eyes respond with about 2.2 gamma. There is an electronic circuit which naturally does a log response, so if you put that before the ADC then it's gamma corrected.
http://www.maxim-ic.com/appnotes.cfm/an_pk/3611
some Kodak sensors
http://www.kodak.com/US/en/dpq/site/SENSORS/name/ISSInterlineProductFamily

As an example, the sensor is 69db range, or about 11bits.
So, I'm saying this is a true, gamma corrected 11bits.

Cineon I assume is used with digital intermediates:
1. Conversion of 10-bit Log Film Data To 8-bit Linear or Video Data, Kodak Picture & Television Imaging, 1995
2. Grayscale Transformations of Cineon Digital Film Data, Cinesite Digital Film Center, 1993

Well, it looks like they are doing a pretty normal quantization. They either chip off 2bits or apply reduced contrast at the extreme ranges (soft clipping).

Anyhow, when I say I might be able to do HDR I'm not talking the crazy 16bit that you can do with fstops or even neutral density filters, I just mean anything over 8bits. And I still believe I can get just a bit more, but I will have to prove it. Removing dark frame and calibrating sounds like a simple script.
http://www.dotcsw.com/doc/cineon1.pdf
http://www.dotcsw.com/doc/cineon2.pdf

[IanB]

To do 24 Bit RGB (3x8bits) to a YUV format and then back again losslessly requires at least 10 bits of resolution in the Y channel.

[jmac698]

Yes; professional video is always 10bits.

The Digital Betacam format records a DCT-compressed component video signal at 10-bit YUV 4:2:2 sampling in PAL (720×576) or NTSC (720×486) resolutions at a bitrate of 90 Mbit/s plus 4 channels of uncompressed 48 kHz PCM-encoded audio. A 5th audio track is available for cueing, and a linear timecode track is also used on the tape.
http://en.wikipedia.org/wiki/HDCAM

Sensors are naturally rgb, they get converted to 10bit yuv.

[morsa]

Not exactly.
Digibeta and HDCAM SR are 10 bit.
Plain HDCAM is 8 bit YUV 3:1:1, DVCproHD is 8 bit, DVCpro50 is 8 bit, D1 was/is 8 bit...

Anyway having an internal precion of higher than 8 bits would be the best and natural path for avisynth to follow in my opinion. As an example try using Avisynth internal levels filter and gamma correction and compare its results with Virtualdub's internal....

[G_M_C]

To do 24 Bit RGB (3x8bits) to a YUV format and then back again losslessly requires at least 10 bits of resolution in the Y channel.

Don't know if i remember correctly; But wasn't the AviSynth development-team planning things like YUV4:4:4 colorplane support/ 16 bit per color RGB = 48 bits per pixel, for some future release ?

That would be one of the first basic steps to realise this kind of HDR. But with YUV4:4:4 support, I think HDRAGC () could even come close to the results "true HDR" gives

[cc979]

i remember reading somewhere, that men in black was recorded with hdr

[IanB]

@G_M_C,

Avisynth 2.6 has support for YUV 4:4:4 (YV24), this still has an 8 bit Y channel, just no chroma subsampling. So RGB -> YUV -> RGB will still be lossey by the same amount as now.

The 3.0 team have the infrastructure in place to do any bits/pixel but I haven't seen any 15 bit/pixel code.

MMX and SSE does not readily lend itself to operations on 16 bit data with the same willingness as with 8 bit data. It certainly is doable, it just moves the shrewdness/cleverness bar to the next level.

The other hurdle is that Avisynth 2.x is based on the VFW AVI interface, which as far as I know has no current hacks to present greater then 8 bit video data. So in theory if we did do any 15 bit/pixel implementation and even if someone wrote a HDCAM_SR_Source() or DigiBetaSource() or someone jazzed Mpeg2Source(), it would only be available internally, we have no way to serve substantially better data to other applications, the best to hope for would be max quality 8 bit RGB.

If you are aware of anybodies plans to do hi res video thru VFW this forum is the right place to bring it to attention.

[morsa]

IanB:

You said nothing about Directshowsource(). What is the actual situation with it?
Wouldn't it be better to concentrate more on using Directshow instead of hacking VFW ?

As an example. I could use Blackmagic's codecs through Directshow.They are 4:2:2 10 bit. Premiere supports it and also has internal support for 10 bit right now. So if you are working with BlackMagic/Decklink you can process all the way in 10 bit and output to BalckMagic codec again. BTW you don't need to have a Decklink card to use it.

[IanB]

If a DirectShow graph outputs more than 8 bit's per channel that won't be a problem, same boat as a HDCAM_SR_Source(), a DigiBetaSource() or a jazzed Mpeg2Source().

Premier directly interfaces with the output codec so it has no issue of feeding in the appropriate data. I guess if some encoding app wanted to use the Avisynth env->Invoke(...) interface instead of VFW then it would work, but it would be specific to that app only.

In principle there is nothing to stop VFW from transfering 15 bit data, it just needs a standard, I guess a FourCC to be allocated, that everyone agrees apon and can code to.

As I said "If you are aware of anybodies plans to do hi res video thru VFW this forum is the right place to bring it to attention."

[Mug Funky]

@ morsa: you could also use QTinput for that if it gets an update - big speed boost with that versus dshow/vfw. it'd need to be modified for 10 bits of course.

[G_M_C]

@G_M_C,

Avisynth 2.6 has support for YUV 4:4:4 (YV24), this still has an 8 bit Y channel, just no chroma subsampling. So RGB -> YUV -> RGB will still be lossey by the same amount as now.

The 3.0 team have the infrastructure in place to do any bits/pixel but I haven't seen any 15 bit/pixel code.

MMX and SSE does not readily lend itself to operations on 16 bit data with the same willingness as with 8 bit data. It certainly is doable, it just moves the shrewdness/cleverness bar to the next level.

The other hurdle is that Avisynth 2.x is based on the VFW AVI interface, which as far as I know has no current hacks to present greater then 8 bit video data. So in theory if we did do any 15 bit/pixel implementation and even if someone wrote a HDCAM_SR_Source() or DigiBetaSource() or someone jazzed Mpeg2Source(), it would only be available internally, we have no way to serve substantially better data to other applications, the best to hope for would be max quality 8 bit RGB.

If you are aware of anybodies plans to do hi res video thru VFW this forum is the right place to bring it to attention.

After seeing this thread, and your answer I looked for the reason why this "True HDR"-discussion and the 48 bpp rang a small bell.

It seems it was discussed upon a bit in the HDRAGC() thread and in fact it seeems I raised the question there. In fact I remember putting that question there, because of the exact same reason in this thread ... True HDR in pictures. And (allmost offcourse :D ) the same guys reacted; Mug Funky was all for it, and Foxy Shadis mentioned a recompile or someting ;)

Anyway, I got my info here:
http://forum.doom9.org/showthread.php?p=835955#post835955

Im not aware of a enhancement of the AVI-container (sadly, because 48bpp would be awesome, and seems to be the "future" HD-standard since a few months; See also the new HDMI 1.3 specs., that make transfer of these streams possible).

[smok3]

...and seems to be the HD-standard since a few months; See also the new HDMI specs.).
what standard is that?

just a practical note, a simple way to obtain 'hdr' data is to use a 3d app that supports rendering that way (like lightwave3d) - besides taking lots of raw pictures using your digicam.

p.s. workflow like that is a really nice problem solver, few years ago i did a little snowscape type of animation using lw3d and rendering to 8bpc was not really possible (instant clipping), 16bpc cineons did a good job though (the rendering engine internal bitdepth is even 128 bit irc).

[squid_80]

As an example. I could use Blackmagic's codecs through Directshow.They are 4:2:2 10 bit. Premiere supports it and also has internal support for 10 bit right now. So if you are working with BlackMagic/Decklink you can process all the way in 10 bit and output to BalckMagic codec again.
In principle there is nothing to stop VFW from transfering 15 bit data, it just needs a standard, I guess a FourCC to be allocated, that everyone agrees apon and can code to.
There's a format called v210 which is similar to UYVY but uses 10-bits per sample, 6 pixels per 128 bits with the 2 highest bits of each dword zeroed. Reference. (http://developer.apple.com/quicktime/icefloe/dispatch019.html#v210) CineForm's ProspectHD codec seems to support this format.

[IanB]

Well it seems Apple are ahead of the game yet again. All these formats have registered (by Apple) fourCC's.

But wow what a pain to unpack and repack!