Parallel process images of different sizes

[Q3CPMA]

Hello, I have a simple script to do image processing (manga processing for a target ereaders) via imwri:

import vapoursynth as vs
import muvsfunc as muvs
import nnedi3_resample as nnrs
core = vs.core

c = core.imwri.Read(IN)
c = nnrs.nnedi3_resample(c, c.width * 2, c.height * 2)
# ToDo: write function to compute w/h using a bounding box so that AR is preserved
c = muvs.SSIM_downsample(c, w=1080, h=1440, fulls=True, fulld=True, sigmoid=True)
c = core.imwri.Write(c, "PNG", OUT, overwrite=True)

src.set_output()

which works more or less perfectly (well, actually it doesn't, SSIM_downsample mangles the gamma curve for grayscale input).

But I'd like to process many images using vapoursynth's builtin parallelism. But I see two problems here:
* imwri.Read can read multiple images of different dimensions, but I somehow doubt filters will be happy to get such a clip; am I right?
* imwri.Write can't take a list of paths like Read to choose arbitrary output names. How would I do it with an array of such paths? Split c into clips of 1 frame and apply Write to them?

By the way, is there a way to emulate `convert -colors 16 -dither FloydSteinberg` using vapoursynth? And if someone has such knowledge, is there a better way to upscale manga that doesn't produce too much jank (tried waifu2x, but not too fan of the "inpainting" artifacts) and doesn't require CUDA?

Thanks for reading this.

[_Al_]

to load images process them and save them:
import vapoursynth as vs
from vapoursynth import core
from pathlib import Path
DIRECTORY = r'F:\images'
paths = Path(DIRECTORY).glob("*.png")
def process_image(path):
c = core.imwri.Read(str(path))
#sequence of your process, here it is just simple resize, put whatever you want
c = c.resize.Bicubic(1440,1080)
path_store = path.parent / f'%04d{path.name}'
c = core.imwri.Write(c, "PNG", str(path_store))
c.get_frame(0)

for path in paths:
process_image(path)
you can rename files then, get rid of those four zeros or something, using python
or maybe this would be faster:
from concurrent import futures

with futures.ThreadPoolExecutor(max_workers=8) as exe:
jobs = [exe.submit(process_image, path) for path in paths]
[job.result() for job in jobs]
or more workers if having more cores

[Q3CPMA]

Well, but with this kind of approach, I don't see the point over simply wrapping the original script in GNU parallel or similar. Tried using starmap from the multiprocessing module, but it didn't work.

[_Al_]

I tested multiprocessing I posted , with your filters and it worked. Images are loaded separately.

trying to load images by imwri.Read providing a list of paths and mismatch=True, then there is problems as you mentioned because format, width and height are dynamic, for example there is no clip.width, so there could be ZeroDivisionError errors pop on from used filters

using imwri.Read with given path list, clip could be resized afterwords giving it size and format but then you cannot control letterbox or pillarbox for resizing, as you mentioned planning on this,
so loading images separately seams like an option, like I posted above

btw. to pillarbox or letterbox an image this could be used something like this:
def to_size(clip, width, height):
if width<clip.width or height<clip.height:
#downsize
return clip.resize.Bicubic(width=width, height=height)
else:
#upsize
clip = nnrs.nnedi3_resample(clip, clip.width * 2, clip.height * 2)
clip = muvs.SSIM_downsample(clip, w=width, h=height, fulls=True, fulld=True, sigmoid=True)
return clip

def to_box(clip, width, height):
cw, ch = clip.width, clip.height
modx = 1 << clip.format.subsampling_w
mody = 1 << clip.format.subsampling_h

if width==cw and height==ch:
return clip

size = 1 << clip.format.bits_per_sample
if clip.format.color_family==vs.RGB:
color=[size//16]*3
else:
color =(size//16, size//2, size//2)

if width/height > cw/ch:
w = cw*height/ch
x = int((width-w)/2)
x = x - x%modx
x = max(0, x)
clip = to_size(clip, width-2*x, height)
if x:
return clip.std.AddBorders(left=x, right=x, color=color)
else:
return clip
else:
h = ch*width/cw
y = int((height-h)/2)
y = y - y%mody
y = max(0, y)
clip = to_size(clip, width, height-2*y)
if y:
return clip.std.AddBorders(top=y, bottom=y, color=color)
else:
return clip

[_Al_]

in vapoursynth R62, argument color for AddBorders seams to not work, so if left out, it defaults to black for those borders, otherwise I got white color

[Q3CPMA]

Thanks for the help. About ThreadPoolExecutor, is it actually parallel, since threads in Python are kinda gimped due to the GIL? In the end, I went with Parallel, seems like less pain, even if I pay the cost of slow Python startup and module loading for every image.

[_Al_]

I actually tried multiprocessing modules, something like this:
from multiprocessing import Pool
def main():
with Pool() as pool:
jobs = [pool.map(process_image, paths)]
if __name__ == '__main__':
main()
or multiprocessing module from concurrent:
from concurrent import futures
def main():
with futures.ProcessPoolExecutor(max_workers=8) as exe:
jobs = [exe.submit(process_image, path) for path in paths]
for job in jobs:
job.result()
if __name__ == '__main__':
main()
oddly , both had to be called from that if __name__ == '__main__' block, otherwise it would not wok (on windows),

but speed was not faster than futures.ThreadPoolExecutor, which is odd, yes, threads should not be faster than multiprocessing, I don't know why, I just went with seemingly fastest.

[_Al_]

using imwri.Read with given path list, clip could be resized afterwords giving it size and format but then you cannot control letterbox or pillarbox for resizing
Actually, that could be done too, I have to correct myself, afer loading dynamic clip using imwri.Read and mismatch=True (with dynamic width, height and format) and then getting a regular clip clip with letterbox or pillarbox for each specific frame, because width and height are stored as a frame attributes (not clip attribute, that is dynamic):

import vapoursynth as vs
from vapoursynth import core
from pathlib import Path
##import muvsfunc as muvs
##import nnedi3_resample as nnrs

#input
DIRECTORY = r'F:\images'

#output:
WIDTH = 1920
HEIGHT = 1080
FORMAT = vs.RGB24


class Resize_dynamic_frames:

def __init__(self, attr_clip, dynamic_clip):
self.WIDTH = attr_clip.width
self.HEIGHT = attr_clip.height
if attr_clip.format.name.startswith('RGB'):
matrix = 0
else:
#imwri.Write would not work with YUV though
matrix = 1
self.dynamic_clip = dynamic_clip.resize.Bicubic(format=attr_clip.format.id, matrix=matrix)
self.MODX = 1 << attr_clip.format.subsampling_w
self.MODY = 1 << attr_clip.format.subsampling_h

def to_size(self, width, height, cw, ch):
## #to make it simple for testing, just simple resize
return self.dynamic_clip.resize.Bicubic(width=width, height=height)

## if width<cw or height<ch:
## #downsize
## clip = self.dynamic_clip.resize.Bicubic(width=width, height=height)
## else:
## #upsize, this needs vs.RGBS
## clip = self.dynamic_clip.resize.Bicubic(width=cw, height=ch)
## clip = nnrs.nnedi3_resample(clip, cw * 2, ch * 2)
## clip = muvs.SSIM_downsample(clip, w=width, h=height, fulls=True, fulld=True, sigmoid=True)
## return clip

def to_box(self, n):
cw = self.dynamic_clip.get_frame(n).width
ch = self.dynamic_clip.get_frame(n).height

if self.WIDTH==cw and self.HEIGHT==ch:
return self.dynamic_clip.resize.Bicubic(width=cw, height=ch)
if self.WIDTH/self.HEIGHT > cw/ch:
w = cw*self.HEIGHT/ch
x = int((self.WIDTH-w)/2)
x = x - x%self.MODX
x = max(0, x)
if x:
clip = self.to_size(self.WIDTH-2*x, self.HEIGHT, cw, ch)
return clip.std.AddBorders(left=x, right=x)
else:
h = ch*self.WIDTH/cw
y = int((self.HEIGHT-h)/2)
y = y - y%self.MODY
y = max(0, y)
if y:
clip = self.to_size(self.WIDTH, self.HEIGHT-2*y, cw, ch)
return clip.std.AddBorders(top=y, bottom=y)

return self.dynamic_clip.resize.Bicubic(width=cw, height=ch)

paths = list(Path(DIRECTORY).glob("*.png"))
dynamic_clip = core.imwri.Read(paths, mismatch=True)
attr_clip = core.std.BlankClip(width=WIDTH, height=HEIGHT, format=FORMAT, length=len(paths))
resize = Resize_dynamic_frames(attr_clip, dynamic_clip)
clip = attr_clip.std.FrameEval(resize.to_box)

for n, path in enumerate(paths):
c = core.imwri.Write(clip, 'PNG', filename=str(Path(path).parent /f'%04d{path.name}'), overwrite=True)
c.get_frame(n)
format has to be RGB or 444 if having odd resolutions, but imwri.Write would work only with RGB for png anyway

Also speed using separate loading and multiprocessing seems to be faster than this, it is going in a linear fashion frame after frame, no parallel process