VirtualDub2

[kolak]

When h265 HDR clip is properly encoded then HDR metadata should be present at every GOP. Some old clips made with x265 may not follow this rule. Current (few months old) x265 has this fixed and adds HDR metadata every GOP by default.

[Revan654]

Thanks. Following Revan654 question I was thinking what can be done with metadata capture. Because it can be a sequence of unrelated clips it must be possible to preserve variable metadata (per scene) despite its static nature. The fact that it is included per GOP helps this idea.


After allot of research, some things I've come up with. This should help. Atlease a starting point.

Shaders looks to be where color space is refined at. Which needs to add BT.2020 to it.


Review source Code(The Original Virtual).

I cam across this: AVIStreamHeader_fixedhdr in the AVIReadHandler.cpp

I have no idea what this means Since I don't understand C Programming.


More noting HDR:


if (hdr.biWidth != mWidth || hdr.biHeight != mHeight)
return false;

const VDAVIBitmapInfoHeader& hdr = *(const VDAVIBitmapInfoHeader *)format;

if (hdr.biWidth != mWidth || hdr.biHeight != mHeight)
return false;

int pxformat = VDBitmapFormatToPixmapFormat(hdr);

return QueryTargetFormat(pxformat);

bool VDVideoDecompressorDIB::SetTargetFormat(const void *format) {
const VDAVIBitmapInfoHeader& hdr = *(const VDAVIBitmapInfoHeader *)format;

if (hdr.biWidth != mWidth || hdr.biHeight != mHeight)
return false;

int variant;
int pxformat = VDBitmapFormatToPixmapFormat(hdr, variant);
if (!pxformat)
return false;

There allot more with HDR in the file, It's all from VideoSource.cpp file.


Also BlackMagic SDK seems to have HDR and Color Space. Maybe that would help speed things up with HDR aspect? Magewell SDK is being released in one to two weeks time.


Some Free Source code Color/ Shaders which has BT.2020 in them already.

Link: https://github.com/Microsoft/DirectX-Graphics-Samples/blob/master/MiniEngine/Core/Shaders/ColorSpaceUtility.hlsli

VDub Filter for BT.2020: https://github.com/jpsdr/Filtres_JPSDR/tree/master/ColorSpaceConvert

Here are allot of Links talking about the MetaData and HDR:

https://msdn.microsoft.com/en-us/library/windows/desktop/mt732708(v=vs.85).aspx

Link 2 (Color Space) : https://msdn.microsoft.com/en-us/library/windows/desktop/dn903661(v=vs.85).aspx

Link 3(Setting HDR): https://msdn.microsoft.com/en-us/library/windows/desktop/mt732708(v=vs.85).aspx

Link 3 (Defining HDR) : https://msdn.microsoft.com/en-us/library/windows/desktop/mt732701(v=vs.85).aspx

Link 4(HDR Structure): https://msdn.microsoft.com/en-us/library/windows/desktop/mt732700(v=vs.85).aspx

Link 5 Wide Range: https://msdn.microsoft.com/en-us/library/windows/desktop/mt742103(v=vs.85).aspx

[shekh]

New update (41768): most effort went to fixing bitdepth scaling errors/inconsistencies.
Using VapourSynth resize as reference, I now confirm bit-to-bit identical results at every sample that I checked.
Previously there were inaccuracies like gray CbCr->0x8080 and inexact scaling of limited range values.

[GMJCZP]

I still try to open a file with the context menu and start VDub 1.10.4, when could this be fixed?

[shekh]

I still try to open a file with the context menu and start VDub 1.10.4, when could this be fixed?

All I can do now is keep this in requests list, so it doesn't get lost. https://sourceforge.net/p/vdfiltermod/tickets/125/

I implement things that I select. How else would you expect? Right now, solving context menus is not on top of my wish list. I'm sorry but it can last long.

[Asmodian]

New update (41768): most effort went to fixing bitdepth scaling errors/inconsistencies.
Using VapourSynth resize as reference, I now confirm bit-to-bit identical results at every sample that I checked.
Previously there were inaccuracies like gray CbCr->0x8080 and inexact scaling of limited range values.

Thanks! Great to see the math getting polished. :)

[fAy01]

Thanks for fixing the conversion errors. Cheers!

[isidroco]

I still try to open a file with the context menu and start VDub 1.10.4, when could this be fixed?

You may try renaming Vdub v1.10.4 (ie: VDUB4.EXE) so context menu won't find it.

[Stereodude]

How about a pie in the sky feature request... An option to use MadVR in a window as the output display for VD2 instead of the standard Windows display API?

[LigH]

A prerequirement, using DirectX (9 or 11) for output panes, seems to be supported already; now "just" select a renderer?!

[shekh]

How about a pie in the sky feature request... An option to use MadVR in a window as the output display for VD2 instead of the standard Windows display API?

Sounds great, and a lot of work.
I expect the fact that I'm already using DX9 has zero help with it

[ricardo.santos]

Hi shekh, thanks for all the work you have put on VirtualDub2/VirtualDubFilterMod, when using VirtualDub2 to only cut/export 1080p videos to cineform i noticed the exported clips are being recognized as being 720p in premiere pro, the exported files are 1080p.

i use the following settings:
SAVE AS: Quicktime

Cineform Configuration:
YUV 4:2:2 10 bit depth
Intermediate Bit Depth: 10
Quality: High

Its not a big problem just wondering if its something im doing wrong
Thanks

[ricardo.santos]

Here's a small 2 sec sample, if i try to create a "timeline/project" on premiere it sets the timeline as 720p, all the clips are zoomed in not showing 33% of the video, but the videos are 1080p, its just premiere pro "video inspector" showing it as 720p.
https://www.dropbox.com/s/jatdtq15p3tcta5/cineform1080p_2sec.mov?dl=0

Its not a big problem since i manually changed the "timeline/project" settings to 1080p and set each video to a 67% zoom level in order for the videos to fill the 1080p area, im not a "coder" but wondering if virtualdub2 is writting any wrong metadata that could "fool" premiere pro?

Thanks

[poisondeathray]

Here's a small 2 sec sample, if i try to create a "timeline/project" on premiere it sets the timeline as 720p, all the clips are zoomed in not showing 33% of the video, but the videos are 1080p, its just premiere pro "video inspector" showing it as 720p.
https://www.dropbox.com/s/jatdtq15p3tcta5/cineform1080p_2sec.mov?dl=0

Its not a big problem since i manually changed the "timeline/project" settings to 1080p and set each video to a 67% zoom level in order for the videos to fill the 1080p area, im not a "coder" but wondering if virtualdub2 is writting any wrong metadata that could "fool" premiere pro?

Thanks

This is 1920x1080 in premiere for me.

It might be you are using older PP version ?

If you are on windows, using older PP version, try using cineform AVI instead

[ricardo.santos]

Thanks a lot poisondeathray!!

created an avi cineform file and premiere recognized it properly, after that i tried some of the MOV videos that were problematic and now they are "tagged" correctly...go figure...im using premiere 2017.

:thanks:

[poisondeathray]

Thanks a lot poisondeathray!!

created an avi cineform file and premiere recognized it properly, after that i tried some of the MOV videos that were problematic and now they are "tagged" correctly...go figure...im using premiere 2017.

:thanks:

Not sure, I just checked on another computer with CC2015 and it's 1920x1080 too . So both CC2018 and CC2015 are reading the MOV correctly . Might be a system problem or corrupt install on your end ?

[ricardo.santos]

I recently discovered that by using cineform videos on premiere pro i could preview the fx/titles/transitions etc without stuttering, so i quickly installed old quicktime codecs (quicktime alternative because of the mov format) and installed an old gopro studio version and the latest Quik software from gopro thinking i could do a "speed" course on the format but after a lot of problems i sat down and calmly reading articles online i found out the format is now open source and natively supported by premiere pro, i had installed a lot of stuff that gave me problems without really needing it.

Since Premiere pro slows down my pc a bit when using a lot of clips and editing, i needed a tool that i could use to just cut and export to cineform and then use premiere a lot less and without stuttering while previewing all the edits....thats how i got to virtualdub2 and to here.

I must do a backup and a fresh install of windows...lesson learned...dont try to rush things

Thanks

[shekh]

Hi,
I'm not attaching any metadata. But the cineform db is quite complex, it has global overrides which may interfere with any cineform file.

[Revan654]

I believe cineform in VD2 is still listed as experiential.

----------

Any chance we will be seeing x265 (Fully Supported, All Flags and options)? or x265 with Nvidia GPU enabled?

------

Also HDR SDK for Magewell has been Released Publicly: http://www.magewell.com/files/Magewell_Capture_SDK_3.3.1.719.zip

Not sure if you still going to add Rec.2020 and HDR10 Support or not.

[shekh]

I believe cineform in VD2 is still listed as experiential.


And the reason is?
I have no reports or suggestions about cineform.

> Any chance we will be seeing x265 (Fully Supported, All Flags and options)? or x265 with Nvidia GPU enabled?

Currently I don't plan it. x265 with Nvidia: this thing does not even exist afaik.

> Not sure if you still going to add Rec.2020 and HDR10 Support

I prefer to message when it's done.

[Revan654]

And the reason is?
I have no reports or suggestions about cineform.

> Any chance we will be seeing x265 (Fully Supported, All Flags and options)? or x265 with Nvidia GPU enabled?

Currently I don't plan it. x265 with Nvidia: this thing does not even exist afaik.

> Not sure if you still going to add Rec.2020 and HDR10 Support

I prefer to message when it's done.

x265 with Nvida does exist. It's just NVEnc, the SDK I uploaded contains examples of it.

If you ever add HDR x265 is required along with certain Flags.

[LigH]

How many more times? Again and again, over and over...

If you let Nvidia's PureVideo chip encode HEVC video, it is not x265. The point of x265 is the complexity of the calculations PureVideo can never provide. NVEnc may produce HEVC video. But it will never be x265.

[wonkey_monkey]

Are we talking about x265 as in the specific encoder here, or using it to refer to HEVC/H.265? Just trying to work out what everyone's saying...

[LigH]

That's the important difference.

There are many HEVC / H.265 encoders out there. But not all of them can be implemented into VirtualDub2, based on libraries available to ffmpeg (e.g. most of them are not available as source code): x265 is one of them, but it will not use any GPU encoding. GPU encoders like NVEnc may be available too, where the hardware supports it, but it won't compete with the efficiency (quality retention per bitrate) of x265.

HEVC encoding in a GPU is way too limiting: less available RAM to store many frames to compare with each other to find similarities; slow uploading of the frames from system RAM into video RAM; limited set of instructions in the GPU, optimized rather for 3D graphics than for video compression ... using SSE, AVX and other modern SIMD instruction sets, and having possibly more than 8 GB RAM available with rather fast access, is much more interesting for video compression with good quality preservation, which requires a higher complexity in the encoding process.

[Revan654]

That's the important difference.

There are many HEVC / H.265 encoders out there. But not all of them can be implemented into VirtualDub2, based on libraries available to ffmpeg (e.g. most of them are not available as source code): x265 is one of them, but it will not use any GPU encoding. GPU encoders like NVEnc may be available too, where the hardware supports it, but it won't compete with the efficiency (quality retention per bitrate) of x265.

HEVC encoding in a GPU is way too limiting: less available RAM to store many frames to compare with each other to find similarities; slow uploading of the frames from system RAM into video RAM; limited set of instructions in the GPU, optimized rather for 3D graphics than for video compression ... using SSE, AVX and other modern SIMD instruction sets, and having possibly more than 8 GB RAM available with rather fast access, is much more interesting for video compression with good quality preservation, which requires a higher complexity in the encoding process.

There is x265VFW, However it's limited when it comes to pixel type.

Maybe can just use the GPU as acceleration option. Since x265 can be taxing on a system.

Or an option to use external encoder while in capture mode. Right now all the codecs are pretty much useless to me since I need to add additional flags to the codecs and there no option for that (Except for x264).

Are we talking about x265 as in the specific encoder here, or using it to refer to HEVC/H.265? Just trying to work out what everyone's saying...

Just a reference to HEVC.

[Yanak]

Hello,

I'm trying to use a mkv containing a .hevc 4.2.0 10bits file inside virtualdub2, and exporting via the frameserver to a .vdr file and then trying to open it in Staxrip.

The .vdr file imports in Staxrip, i can see the length of the file being correct but all the frames are returned black, until now i always used 8bits videos and with those there is no problems using avisource in avisynth+ for .vdr files.

Vdub2 in automatic mode for the decode format sets it to YUV420P16 apparently ( file information in vdub2 says yuv420p10le ), tried to set it manually to YUV420P10 but still black frames returned in avisource, if i set the decode format to RGB24 it returns the frames normally, not black i mean, but at this point i didn't tried to compare colors vs original frames to see if there is some errors.

I saw there is the possibility to set the pixel_type for avisource but trying YUV420P16 or YUV420P10 still return black frames on the .vdr file, tried also to use things like fourCC="Y3[11][10]" in the avisource filter but seems to have no effect, still black frames.

I'm a bit lost in this now and not sure if it comes from avisynth or from the .vdr file exported by the frameserver and possible problems with 10 bits content,

If anyone have an idea about this please,

Thanks a lot.

[shekh]

Hello,
I see this problem, investigating.

Frameserving always uses RGBA output format. Currently you can fix it by adding a filter "convert format (RGBA32)". There is nothing to do on avisource side.

edit: actually, frameserver outputs RGB24 but the workaround is good anyway.
edit2: fixed in new build.

[Yanak]

Hello,

Thank you very much for the quick reply, (sorry for the late reply, had to go out in emergency just after posting this yesterday), i understand now, will try the new build and use convert to rgb24.

Thanks again for all :)

[color]

I did use Virtualdub2 for a few days without any problems, but today, all videos is shown in the left preview window, but the right one is grey. I have not done anything with the codecs or changed any options. Tried to uncheck DirectX and restarted the program and windows without any luck.

[shekh]

I did use Virtualdub2 for a few days without any problems, but today, all videos is shown in the left preview window, but the right one is grey. I have not done anything with the codecs or changed any options. Tried to uncheck DirectX and restarted the program and windows without any luck.

Have you unchecked "View->Output video pane"?

[color]

sekh, oh, haha. Yes it was unchecked. I saw It was "F10" Its the same button to change audio on my keyboard but pressing "fn"-button, I guess that might be why. Thank you. :)

[kolak]

Could we have bayer 12bit pixel format added, so after loading CR RAW files we could render them (eg as RGB 12 or 16bit) keeping Bayer pattern? This would need bypass Cineform own debayering, but I know there is such an option in Cineform SDK.

[lansing]

Hi I want to track down an annoying bug in capture mode myself, I cloned the project and opened the VirtualDub.sln with Microsoft Visual Studio Community 2017. How do I build it for testing? I choose "build solution" but it failed with message about "build tool for v140_xp can not be found". What else do I need?

Update: I found the solution here (https://developercommunity.visualstudio.com/content/problem/48806/cant-find-v140-in-visual-studio-2017.html), I can install it through the visual studio installer, but I don't have 5 GB in my hard drive...is there a way to get around this?

[lansing]

I filed a ticket (https://sourceforge.net/p/vdfiltermod/tickets/161/)for this issue instead, I just couldn't squeeze out enough hard drive to install all the programs.

[shekh]

Hi I want to track down an annoying bug in capture mode myself, I cloned the project and opened the VirtualDub.sln with Microsoft Visual Studio Community 2017. How do I build it for testing? I choose "build solution" but it failed with message about "build tool for v140_xp can not be found". What else do I need?


These instructions should work: https://github.com/shekh/VirtualDub2/blob/master/BUILDING.txt

It is not necessary to use v140_xp toolset, you can change it to v140 or whatever is already installed. It will not run on XP but you should not care. See VirtualDub project properties, general, toolset.

Also replied to ticket.

[shekh]

Could we have bayer 12bit pixel format added, so after loading CR RAW files we could render them (eg as RGB 12 or 16bit) keeping Bayer pattern? This would need bypass Cineform own debayering, but I know there is such an option in Cineform SDK.

It is possible. Can you describe in more detail how would you use this?
Do you want to save it again as bayer format, or apply some custom debayering outside VD?

[kolak]

Ideally I would like to convert CinemaDNG RAW into CF RAW :)
Converting TIFF with RAW data into CF RAW would be already cool. I can convert CDNG RAW with dcraw tool into "RAW" TIFF.
Decoding CF RAW and exporting as 12bit uncompressed file keeping RAW look for testing purposes should be easy to implement.

[wonkey_monkey]

This is very minor, but I've noticed that pressing Alt doesn't activate the menu bar as it would in most other Windows software. If I hold Alt and press F, it does open the File menu, but if I separately press Alt, and then F, it doesn't. Is this deliberate?

[shekh]

This is very minor, but I've noticed that pressing Alt doesn't activate the menu bar as it would in most other Windows software. If I hold Alt and press F, it does open the File menu, but if I separately press Alt, and then F, it doesn't. Is this deliberate?

Mistake in hotkey handling, fixing. Thanks for report.

[shekh]

Ideally I would like to convert CinemaDNG RAW into CF RAW :)
Converting TIFF with RAW data into CF RAW would be already cool. I can convert CDNG RAW with dcraw tool into "RAW" TIFF.
Decoding CF RAW and exporting as 12bit uncompressed file keeping RAW look for testing purposes should be easy to implement.

After some experimenting:
"dcraw -6 -E" creates non-interpolated bayer image from dng etc that looks quite the same as CFHD_PIXEL_FORMAT_BYR4 image extracted from cineform. So this part is trivial, and likely no problem to compress such image to cineform raw format. But what about metadata, does cineform raw without any relevant metadata make sense? Translating the metadata could be a challenge.
Also not sure about CinemaDNG but to import DNG sequences it would require a new source plugin, as nothing like that seems integrated with FFMpeg already. Such plugin can be derived from dcraw or from gopro/gpr (maybe from both). This needs some effort, not promising now.

[kolak]

Maybe add -W which should keep RAW untouched (not brightened- not sure).

Metadata is tricky part for sure. Here is some hint:
http://vfx-rants.blogspot.com/2012/08/bmcc-raw-to-cineform-raw.html

I want to use Cineform RAW in Resolve which does bypass Cineform metadata and debayering anyway, so I assume it still make sense.
There are 2 parts in this article:
- make debayered image look LOG, but not green (so you apply inverted embedded color matrix)
- 2nd part is to give it a "final" look for sRGB/REc.709 screen (optional)

Here you have BM Ursa Mini original assets:
http://translate.google.com/translate?sl=de&tl=en&js=n&prev=_t&hl=de&ie=UTF-8&u=http%3A%2F%2Flaufbildkommission.wordpress.com%2F

which represent good source. If you can convert few frames into CF RAW (no metadata) I can have a look.

[shekh]

Here you have BM Ursa Mini original assets:
http://translate.google.com/translate?sl=de&tl=en&js=n&prev=_t&hl=de&ie=UTF-8&u=http%3A%2F%2Flaufbildkommission.wordpress.com%2F

which represent good source. If you can convert few frames into CF RAW (no metadata) I can have a look.

Hope you can deal with it. Looks not too bad, at least details are present.

https://drive.google.com/open?id=1OBHWIoTvImZkXnF2r6X2Q9fPQy6w-9O1

[kolak]

Hm... I had similar results. This is not really good. You can't get "proper" look from it after debayering. Something seems to be wrong.

[shekh]

Hm... I had similar results. This is not really good. You can't get "proper" look from it after debayering. Something seems to be wrong.

How about this one: https://drive.google.com/open?id=1uXbtuI4PZDKKXS41XVI_fRP13i6rq7lV

It seems BM Ursa pattern is GBRG rather than RGGB so I flipped the raw image vertically before encoding. Colors turn legible

[shekh]

Build 42122, bugfix and very experimental CF raw support.

To encode CF raw: open grayscale 16-bit images (I16), any format like ppm,pgm,tiff is ok. Select "bayer" in codec settings.
To decode CF raw: select decode format->grayscale. This triggers bayer output in I16 format.
It appears that there is conversion error between I16<->RGBA64 (clipped dark values)

[kolak]

Great! It's fine now, except image is 180 degrees flipped :)
Not sure how, but now if you could just pass to CF RAW this matrix values to bring image into more similar look to DNG.

[shekh]

Great! It's fine now, except image is 180 degrees flipped :)

It's just to confirm the workflow :) Now it needs some simple filter to change mosaic order instead of flipping.
Another alternative with this source is to crop 1 line off top and bottom.

[kolak]

I'm playing with it now.
Footage looks quite fine in Resolve. Any colorist can grade from it (even without all those somehow helpful matrixes applied).
Pattern will be different for different camera makers, so this is something to "sort out"- eg. have setting to force specific one.

My method was very convoluted. Save dcraw decoded TIFF with ffmpeg as uncompressed 16bit raw video with BYR4 header (which allowed CF encoder to link to it and switch to RAW mode) and then use diretchsow encoder in grpahedit :) It did work though :)

[kolak]

Trying to measure CF RAW (to source RAW) PSNR and having issue with levels.

"It appears that there is conversion error between I16<->RGBA64 (clipped dark values)"

I assume this is an issue. I can't at the moment save decoded CF RAW into something which I could compare against original TIFF RAW.

[kolak]

How about this one: https://drive.google.com/open?id=1uXbtuI4PZDKKXS41XVI_fRP13i6rq7lV

It seems BM Ursa pattern is GBRG rather than RGGB so I flipped the raw image vertically before encoding. Colors turn legible

Yes, dcraw confirms that it's RGRB.