ImageSourceNV plugin

[Argaricolm]

I'v made a plugin that uses nvjpeg api to decode jpeg source files and convert their color space to YUV444. That is CUDA accelerated.
You will need CUDA toolkit 12.4 (or 11.8) installed and MS C++ redistributables.

Usage:
ImageSourceNV("C:\Folder1\Folder2","jpg",1,34609,24000,1001)

1: Folder path without ending slash.
2: Files extension.
3: First frame.
4: Last frame (included).
5: FPS numerator.
6: FPS denumerator.

File names should be 6 numbers with leading zeroes (%06d).

Download link (https://disk.yandex.ru/d/ViCV8bgqSZltcA)

Update 1:
Added two new CUDA functions:
ConvertToRGBNV()
ConvertToYUVNV()

ConvertToRGBNV function converts YUV444 or YUV420 to RGB 32 bit (technically BGR in avisynth).

ConvertToYUVNV converts RGB32 to YUV420.
I'v made them because of the need to use RGB filters.
YUV<->RGB conversion is done with formula published on MS (https://learn.microsoft.com/ru-ru/windows/win32/medfound/recommended-8-bit-yuv-formats-for-video-rendering)site.
Nvidia NPP functions use different formula, that makes incorrect (brighter) RGB results and also outputs BGR pixels in reversed order, that makes image flipped. So they were not used.

Update 07.07.2023:
Fixed one bug that caused last (bottom) row to be black line in convert RGV<->YUV functions.
Recompiled with CUDA 12.2.

Update 22.04.2024:
Changed YUV<->RGB conversion formula. Previous one was only for limited color range. Now it will work for full color range too.

Update 04.10.2024:
1) Added option for ConvertToRGBNV function called "full".
Now you can convert from YUV to RGB without changing color range, so it will be limited range in RGB (if YUV was in limited).
This is useful if you don't need full range.
Like for example if you use some denoising RGB function you just do:
ConvertToRGBNV(full=1)
Denoise()
ConvertToYUVNV()
This way you will not do color range conversion twise.
2) ImageSourceNV function was fixed. The one before was returning a full range YUV. Now it will decode jpeg to RGB, then change range to limited, and then convert to YUV. So returned YUV will be in limited range. Rec.709 is used here for RGB->YUV conversion.

[kedautinh12]

your plugin for x86 or x64??

[Reel.Deel]

It's x64.

@Argaricolm

Thank you for the plugin. Any chance of a CUDA 11.8 compile for us Windows 7 users?

[Selur]

A VapourSynth version would be nice. :)

[Reel.Deel]

If you're reluctant to install the 3GB CUDA toolkit, you can the individual files also. According to DependencyWalker it only needs these two files:

cudart64_12.dll: https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/windows-x86_64/
nvjpeg64_12.dll: https://developer.download.nvidia.com/compute/cuda/redist/libnvjpeg/windows-x86_64/

[Argaricolm]

But these 2 dlls are surely have other dependencies...
Added CUDA 11.8 version. Redownload archive.

[Reel.Deel]

But these 2 dlls are surely have other dependencies...
Added CUDA 11.8 version. Redownload archive.

Tried out the 11.8 version but it seems that it requires nvjpeg from Cuda 12?

https://i.ibb.co/hBCP27L/imagesourcenv118.png

If not too much trouble, maybe a static build would be better.

[Argaricolm]

Tried out the 11.8 version but it seems that it requires nvjpeg from Cuda 12?


Fixed. Redownload.
Made static linking. But it links only C++ library functions. Cuda is still needed.

[Reel.Deel]

Fixed. Redownload.
Made static linking. But it links only C++ library functions. Cuda is still needed.

Thanks for the update. I got it to run but the output is only a green image:

ImageSourceNV("E:\Images", "jpg", 1, 6, 24000, 1001)
https://i.ibb.co/zSyXBJF/nvtest.png

Dimensions, frame count and frame rate are correct, though.

I did have to fetch two additional dependencies for cudart64_110.dll (api-ms-win-core-libraryloader-l1-2-0.dll and api-ms-win-security-systemfunctions-l1-1-0.dll). I'm not sure why they're missing or if they are meant for Win10.

[Argaricolm]

Image width & height are got using nvjpeg. So - it is working.
I'v tested it with 1048 height and working here.
What software are you using to open avs? And which version of avisynth?

[Emulgator]

My main system is down so I can only remote guess from a replacement system below the specified abilities:
The given commandline does not specify spaces and wants to see 000000.jpg, 000001.jpg ?
And any green frame would usually be YUV samples decoded as 0,0,0 (invalid, btw.)

[Argaricolm]

Yes, it needs file names: 000001.jpg
6 numbers. Same as in input parameters.
It gets frame settings from first jpeg.

[Reel.Deel]

Image width & height are got using nvjpeg. So - it is working.
I'v tested it with 1048 height and working here.
What software are you using to open avs? And which version of avisynth?

I'm using AviSynth 3.7.3 test 9 (https://forum.doom9.org/showthread.php?p=1984568#post1984568) and tested with VDub2 and AvsPmod and both show the same result. Image is 4:2:0.

I also noticed that ImageSourceNV("E:\Images", "jpg") results in "ZeroDivisionError: float division by zero".

[Emulgator]

System up again, success.
Threw the cuda 11.8 .dlls into system32, used the 11.8 plugin version.

From non-subsampled .jpg source filter returns frames decoded to YV24, 23.976fps
Tested folder with 5 frames starting 000001.jpg...000005.jpg
returns the 1st frame 000001, then repeats the same frame until framecount.

Tested folder with 5 frames starting 000000.jpg...000004.jpg
returns frame 000000, then 000001, then 000002, then repeats 000002, then 000000.
(My bad: I had given some pictures with differing width/height, first 3 were portrait, then 000003 was landscape X-}
00000.jpg: No video

Now a folder renamed to 000000.jpg to 000200.jpg, all same dimensions 4608x3456, orientation landscape.
These 4:2:2-subsampled .jpgs result in chroma planes with half width,
so resizing would be necessary afterwards, or better: chromaspace signalling should match.

ImageSourceNV() returns all frames decoded with unresized chroma planes to YV24, 23.976fps.

With prefetch(16,200) 4k frames can be decoded almost in film speed (RTX3080 @ 15..30%).
Not bad for flipping through albums.

[Reel.Deel]

I used the oldest versions available (cudart 11.4.108 / nvjpeg 11.5.2.120) and now it works for me. I will go back and see what is the latest version that works for me since I'm on Windows 7. I think the problem may be that the newest 11.8 cudart requires api-ms-win-core-libraryloader-l1-2-0.dll and api-ms-win-security-systemfunctions-l1-1-0.dll while the oldest one does not.

Edit: the latest cudart that does not require those additional dependencies is 11.6.55.

[Emulgator]

Thank you, good to know.

[Genji]

In an application called Amatsukaze (internally using Avisynth Neo), writing ConvertToRGBNV() immediately after OnCUDA(2) results in an error.

avs example 1 -> exits with error
...
KFMDeint(xxxxx)
OnCUDA(2)
ConvertToRGBNV()
xxxxxxx
ConvertToYUVNV()
OnCPU(2)
...

Output Error
Message: [CUDA Error] 77: an illegal memory access was encountered @919

ConvertToYUVNV() is processed normally without error.

avs example 2 -> successful completion
...
KFMDeint(xxxxxxx)
OnCUDA(2)
ConvertToYUVNV()
xxxxxxx
OnCPU(2)
...

Is this due to Amatsukaze(Avisynth Neo) or something different between ConvertToRGBNV() and ConvertToYUVNV()?

[Argaricolm]

I have not used Amatsukaze
Probably you should use OnCUDA(2) after Convert.
Convert functions work internally only on CUDA. So no need to specify anything.
If you don't have CUDA support by drivers that functions will not do anything.

[Argaricolm]

A new update. Check top message for info.

[Selur]

Any plans for a Vapoursynth version?